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Summary 


The importance of adolescent literacy is well established, and the topic continues to be of 
both local and national interest. Practitioners need to know not only which programs and 
practices appear effective, but which have the scientific evidence to support that claim. 

To identify effective programs and practices for general education students in grades 6-12, 
this review examined studies published over the past 20 years using What Works Clearing- 
house (WWC) standards (version 3.0; U.S. Department of Education, 2014) to evaluate 
the scientific rigor of their research design. A review of the literature on adolescent literacy 
identified 7,144 studies. Of these, 111 met the criteria that made them eligible for further 
review using the WWC standards. Thirty-three of the 111 were determined by the review 
team to have met WWC evidence standards with or without reservations. Twelve of the 
33 studies — each representing a different program or practice — were identified as having 
positive or potentially positive effects on reading comprehension, vocabulary, or general 
literacy. 

The following are key findings from the 12 programs and practices demonstrating such 
effects: 

• Most of the identified programs and practices included instructional elements such 
as explicit instruction in reading comprehension or use of instructional routines. 

• These programs and practices can be implemented within the structure of a typical 
middle-school language arts or content-area classroom. In most cases implementa- 
tion involved ongoing support for teachers. 

• None of the 12 identified programs and practices was conducted in a high school 
setting. 

• Some of the programs and practices were identified as having potentially positive 
effects on high-stakes outcome assessments such as state accountability reading 
assessments. 

Continuing rigorous research on currently available programs and practices is needed to 
expand understanding of their effectiveness. Additional research is also needed in adoles- 
cent literacy to expand the number and types of programs and practices available. 
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Why this review? 


Despite increased public attention to low reading proficiency rates and minimal improve- 
ment, adolescent literacy continues to be a topic of local and national interest because the 
problem persists. The 2013 National Assessment of Educational Progress report shows that 
in 2013 only about 36 percent of students in grades 8 and 12 scored at or above the profi- 
cient level in reading. In addition, the report reveals only a three-point increase in reading 
proficiency for students in grade 8 since 2011 and no significant change for students in 
grade 12 since 2009 (National Center for Education Statistics, 2013). To improve these 
trends, more evidence is needed on effective literacy instruction for adolescents. 


There is already a large body of research on adolescent literacy instruction, ranging from 
specific, targeted one-on-one interventions to small-group pullout instruction, and from 
classroom-based content-area reading to broad, whole-school approaches. But it is import- 
ant for practitioners to know what programs and practices have rigorous scientific evidence 
to support their effectiveness. That can be determined only through research studies that 
demonstrate rigorous research designs. 

The rigor of a research design can be evaluated by focusing on several key components. 
They include the way students are assigned to treatment and comparison groups, the 
number of students from each group who do not complete the study, the equivalence of 
treatment and comparison groups before treatment was implemented, and the presence 
of other variables that may help explain differences between groups (such as an outcome 
assessment that is collected in different ways for different groups). How well a study can 
prove the effectiveness of a program or practice depends on how well each research design 
component is executed. 


How well a study 
can prove the 
effectiveness 
of a program or 
practice depends 
on how well each 
research design 
component is 
executed 


The What Works Clearinghouse (WWC) provides standards for assessing the rigor of 
research design (U.S. Department of Education, 2014). Studies that meet these standards 
can offer scientific evidence of the effectiveness of a given program or practice. 

What this review examined 


The current review used a systematic process modeled after the WWC Adolescent Literacy 
Protocol Version 3.0 (see appendix A) to answer the following research question: 

• Among programs and practices studied in the past 20 years that are intended to 
improve reading comprehension, vocabulary, or general literacy, which are effec- 
tive for adolescents in grades 6-12 (ages 11-18)? 

Grades 6-12 were included because they are the grade levels typically associated with 
middle and high school configurations. See appendix A for information on the search 
procedures. 

Studies identified in this review include those determined by the review team to have met 
WWC standards with or without reservations. 1 Programs and practices described in these 
studies are categorized as demonstrating positive effects, potentially positive effects, mixed 
effects, no discernible effects, potentially negative effects, or negative effects (table 1). The 
current review summarizes considerations for implementation and common instructional 


1 


Table 1. What Works Clearinghouse terminology used for rating study and 
intervention effects 


Effect rating Definition 


Study effect ratings 


Statistically significant 
positive effect 

The treatment group performed better than the comparison group by a 
statistically significant margin. 

Substantively important 
positive effect 

The treatment group performed better than the comparison group by a 
margin that is important for practical application but did not reach statistical 
significance. Practical importance is determined by an effect size of .25 or 
higher. 

Indeterminate effect 

The treatment group and the comparison group performed at about the 
same level. The difference between the two groups was neither statistically 
significant nor practically important. 

Substantively important 
negative effect 

The comparison group performed better than the treatment group by 
a margin that is important for practical applications but did not reach 
statistical significance. Practical importance is determined by an effect size 
of .25 or higher. 

Statistically important 
negative effect 

The comparison group performed better than the treatment group by a 
statistically significant margin. 

Intervention effect ratings 


Positive effects 

Two or more studies show statistically significant positive effects, with 
at least one meeting What Works Clearinghouse standards without 

reservations. 

Potentially positive effects 

At least one study shows statistically significant or substantively important 
positive effects. 

AND 

The same number of studies or fewer show indeterminate effects than show 

statistically significant or substantively important positive effects. 

Mixed effects 

At least one study shows statistically significant or substantively important 
positive effects. 

BUT 

More studies show indeterminate effects than show statistically significant 
or substantively important positive effects. 

No discernible effects 

None of the studies shows statistically significant or substantively important 
effects in either the positive or negative direction. 

Potentially negative effects 

One study shows statistically significant or substantively important negative 
effects, AND no studies show statistically significant or substantively 
important positive effects. 

OR 

Two or more studies show statistically or substantively important negative 
effects, with at least one displaying statistically significant or substantively 
important positive effects, AND more studies show statistically significant or 
substantively important negative effects than show statistically significant or 
substantively important positive effects. 

Negative effects 

Two or more studies show statistically significant negative effects, with 
at least one meeting What Works Clearinghouse standards without 

reservations. 

AND 

No studies show statistically significant or substantively important positive 
effects. 


Source: U.S. Department of Education, 2014 
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elements across the programs and practices identified as demonstrating positive or poterv 
tially positive effects on reading comprehension, vocabulary, or general literacy outcome 
assessments. 


What this review found 


More than 7,100 studies were identified by the search procedures. Of these, 111 used a 
research design that examined the effectiveness of programs and practices on reading com- 
prehension, vocabulary, or general literacy for general education students in grades 6-12 
residing in the United States and were reviewed by the review team (see appendix B for a 
list of the 111 studies). Eighteen of these 111 studies had been reviewed by WWC but were 
re-reviewed by the review team following the review protocol outlined in appendix A. This 
review found that 33 of these studies had met WWC version 3.0 evidence standards either 
with or without reservations (figure 1; see appendixes C and D for descriptions of the 33 
studies). 

These 33 studies represent 29 programs and practices (see box 1 for definitions of key 
terms). Of these 29 interventions 1 was identified as having a positive effect, 11 as having 
potentially positive effects, 2 as having mixed effects, 15 as having no discernible effects, 
and none as having potentially negative or negative effects (table 2). 

The rest of this section focuses on the 12 programs and practices demonstrating positive 
or potentially positive effects. 2 Considerations for implementation of the 12 programs and 
practices are followed by summaries of the instructional elements common among them. 

The discussion of common instructional elements cannot provide scientific evidence of the 
effectiveness of any one element. Studies that reviewers determined had met WWC Stan- 
dards for research design can only offer scientific evidence of the program or practice as 
implemented in the study; this includes the specific combination of instructional elements 


Figure 1. Screening funnel for adolescent literacy studies reviewed 



studies found in literature search 


met eligibility screening 


met What Works Clearinghouse evidence standards 
with or without reservations 


showed positive or potentially positive effects on student 
reading comprehension, vocabulary, or general literacy 


This review found 
33 studies that 
examined the 
effectiveness 
of programs 
and practices 
on reading 
comprehension, 
vocabulary, or 
general literacy for 
general education 
students in grades 
6-12 residing 
in the United 
States and that 
met WWC version 
3.0 evidence 
standards either 
with or without 
reservations 


Note: The 12 studies that identified programs and practices as having positive or potentially positive effects 
assessed 12 programs and practices. One of these studies included two studies of the same intervention, 
both of which demonstrated statistically significant positive effects (Vaughn et al., 2009). Therefore, this prac- 
tice was classified as having positive effects because it met the criteria described in table 1. 

Source: Authors' compilation. 
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Box 1. Key terms 


Type of program or practice 

• Curriculum. A program that includes all materials (text, lessons, and supporting material) 
for a full course. 

• Lesson package. A set of lessons that may be implemented by a teacher in addition to or 
within another curriculum. 

• Instructional method. A specific set of practices employed by the teacher that provides a 
predictable structure for daily instruction. An instructional method does not necessarily 
need to be associated with specific text or content. 

Support for implementation 

• Implementer certification. Educators who fulfill a set of training requirements are desig- 
nated by the program as certified. 

• Training prior to implementation. One or more workshops were conducted before the start 
of the school year or during a teacher professional development session during the school 
year. 

• Booster sessions. Regularly scheduled meetings (weekly or monthly) conducted to rein- 
force parts of the training conducted before implementation. 

• Coaching. The availability of an expert educator to provide feedback to teachers and prob- 
lem-solve with them. 

• School leadership support. A consultant for the program who is available to the school’s 
curriculum administrator or coach to make structural recommendations or problem-solve. 

Common Instructional element 

• Explicit instruction in reading comprehension. Instruction that includes activities or expla- 
nations of what to do when students don’t understand a text. 

• Explicit instruction in vocabulary. Instruction that includes activities or explanations that 
illustrate the meaning of individual words. 

• Instructional routine. A prescribed, detailed, step-by-step, predictable sequence of 
instruction delivery. 

• Cooperative learning. Students are placed in groups or pairs and assigned a learning task. 

• Feedback. Students are provided with explicit information regarding their performance. 

• Fluency building. Activities designed to promote increasing automatic recognition of words 
in connected text. 

• Writing. Students compose a paragraph or more, which is implemented as an additional 
activity for comprehending text. 

Outcome assessment. An assessment that is administered after the intervention is complet- 
ed. It assesses the skill or ability that the intervention is designed to address (e.g. , vocabulary, 
reading comprehension, general literacy). 

General literacy outcome assessment. An assessment that incorporates two or more of any 
reading-related constructs by providing some type of summary score, such as total reading. 
State outcome assessments or other standardized assessments in which reading comprehen- 
sion is not the only component would be considered a general literacy outcome assessment. 

Source: Authors' creation based on similarities across the descriptions of the programs and practices 
reported in the 12 studies that were identified as having positive or potentially positive effects. 


4 


Table 2. Summary of effects for the 33 studies of adolescent literacy programs and 
practices identified as having met What Works Clearinghouse evidence standards 
with or without reservations 


Findings of the 
effectiveness study 

Program or practice 

Positive effects 3 


Multicomponent social studies instruction (Vaughn et al., 2009) d 

Potentially positive effects 3 


CareerStart (Woolley, Rose, Orthner, Akos, & Jones-Sanpei, 2013) 

Dramatic Impact (Walker, Tabone, & Weltsek, 2011) 

Embedded Story Structure (Faggella-Luby, Schumaker, & Deshler, 2007) 
GeoLiteracy (Hinde et al., 2007) 

Promoting Acceleration of Comprehension and Content through Text (Vaughn, 
Swanson, et al., 2013) 

Read 180 Enterprise (Kim, Capotosto, Harty, & Fitzgerald, 2011) 

Reading Edge (Chamberlain, Daniels, Madden, & Slavin, 2007) 

Read Now (Algozzine, 2004) 

Story Impressions (Denner, Rickards, & Albanese, 2003) 

Student Team Reading and Writing (Stevens, 2003) 

Tutoring (Rothman & Flenderson, 2011) 

Mixed effects 11 


Content Literacy Continuum (Corrin et al., 2012) 

Reading Apprenticeship (Corrin, Somers, Kemple, Nelson, & Sepanik, 2008; 
Greenleaf, Flanson, et al., 2011; Greenleaf, Litman, et al., 2011; Kemple 
et al., 2008) 

No discernible effect 0 


ClassWide Peer Tutoring (Neddenriep, Skinner, Wallace, & McCallum, 2009) 


• Collaborative Strategic Reading (Vaughn et al., 2011; Vaughn, Roberts, et al., 

2013) 

• Components of Intelligent Tutoring of the Structure Strategy (Meyer et al., 2010) 

• Comprehension Circuit Training (Fogarty et al., 2014) 

• Early Start to Emancipation Preparation one-on-one tutoring (Zinn & 

Courtney, 2014) 

• Fast ForWord Language (Borman, Benson, & Overman, 2009) e 

• Fluency building (Allinder, Dunse, Brunken, & Obermiller-Krolikowski, 2001) 

• Renzulli Learning (Field, 2010) 

• Schoolwide Enrichment Model Reading Framework (Little, McCoach, & Reis, 

2014) 

• Single-sex classrooms (Belcher, Frey, & Yankeelov, 2006) 

• Structured homework assignment (Alber, Nelson, & Brennan, 2002) 

• Thinking Reader (Drummond et al., 2011) 

• Tier II instruction emphasizing word recognition, vocabulary, fluency, and 
comprehension (Vaughn, Cirino, et al., 2010; Vaughn, Wanzek, et al., 2010). 

• Title, Examine, Look, Look, and Setting program (Ridge & Skinner, 2011) 

• Xtreme Reading (Corrin et al., 2008; Kemple et al., 2008) 

a. Indicates confidence that there is a real, causal relationship between the program or practice and any 
subsequent changes in student performance and that the probability of observing such a result by chance is 
very slim. For more information, see U.S. Department of Education (2014), WWC Glossary of Terms (http://ies. 
ed.gov/ncee/wwc/Glossary.aspx), and Resources (http://ies.ed.gov/ncee/wwc/Resources.aspx). 

b. Indicates that the evidence of a program’s or practice's effect on student performance in reading compre- 
hension, vocabulary, or general literacy is inconsistent (that is, some results were positive and some were 
indeterminate). 

c. Indicates that there was no evidence that the program or practice had an effect on student performance 
in reading comprehension, vocabulary, or general literacy. For more information, see U.S. Department of 
Education (2014), WWC Glossary of Terms (http://ies.ed.gov/ncee/wwc/Glossary.aspx), and WWC Resources 
(http://ies.ed.gov/ncee/wwc/Resources.aspx). 

d. This study reported findings from two studies, both of which demonstrated statistically significant positive 
effects. Therefore, this practice was classified as having positive effects because it met the criteria described 
in table 1. 

e. The effects for this study were not able to be calculated according to WWC procedures based on the infor- 
mation provided by the authors in the article. 

Source: Authors' literature review (see appendix A). 
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used, the participant sample used, the setting in which the intervention was delivered, 
and the level of implementation. Although this review relied heavily on WWC protocols, 
procedures, and standards and the reviews were conducted by WWC-certified reviewers, 
this review is not a WWC product. 

In addition to the effectiveness of a program or practice, educators must consider other 
practical aspects to implement the program or practice effectively, such as the type of 
program or practice, personnel requirements, setting in which implementation will occur 
(for example, content-area classroom, after-school, and instructional group size), intensity 
and duration of the implementation, and professional development required. This section 
describes such practical considerations in the context of the 12 programs and practices 
identified as demonstrating positive or potentially positive effects. 

The 12 programs and practices identified as having positive or potentially positive effects included 
instructional methods, curricula, and lesson packages 

All 12 programs and practices identified as having positive or potentially positive effects 
included instructional methods, curricula, or lesson packages (table 3). Three of the 12 
programs and practices were considered curricula, and all were commercially available. 
Two of the curricula required access to a computer. Four of the 12 programs and practices 
contained a package of lessons, and three of these lesson packages (for all but Tutoring) 
were commercially available. 

Seven of the 12 programs and practices identified were considered instructional methods. 
They were adapted as a framework for the text or topics used by the school. Availability 
of these instructional methods varies. For methods not commercially available, additional 
information is provided in appendix D. 

Nine of the 12 programs and practices were implemented in an English language arts class- 
room or content-area classroom (such as social studies). The rest were implemented either 
after school or during summer school. 

None of the programs and practices identified as having positive or potentially positive effects has 
been examined in a high school setting, suggesting a clear gap in the literature 

All but one of the programs and practices included students in grades 6-8, a middle school 
setting (see table 3). Only one study demonstrating potentially positive effects was con- 
ducted with students representing a high school population. However, this study took 
place in a summer school program before the start of grade 9 — not a typical high school 
setting. The lack of studies demonstrating a positive or potentially positive effect in a high 
school setting suggests a gap in the adolescent literacy literature. 

A majority of the programs and practices identified as having positive or potentially positive effects 
used primarily whole-class grouping during instruction 

Whole-class instruction was featured in nine of the programs and practices identified 
as having positive or potentially positive effects (see table 3). This grouping preference 
spanned all three program or practice types: curriculum, lesson package, and instructional 
method. Across programs and practices using primarily whole-class instruction, four had 


Three of the 12 
programs and 
practices identified 
as having positive 
or potentially 
positive effects 
were considered 
curricula, and all 
were commercially 
available 
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Table 3. Implementation considerations for the 12 programs and practices identified as having positive and potentially positive effects 




Type of program 
or practice 

Location 





Support for implementation 

Intervention 

Citation 

Commercially 

available 

Curriculum 

Instructional 

method 

Lesson 

package 

Language arts 

classroom 

Content 

area classroom 

Extended 

instruction 

Grade 

Duration/intensity 

Total 

instructional 

time 

(hours) 

Grouping 3 

Implementer 

certification 

Training prior to 

implementation 

Booster 

sessions 

Coaching 

School 

leadership 

support 

Career Start 

Woolley et al. , 

2013 

✓ 


✓ 

✓ 

✓ 

✓ 


6-8 

50-140 minutes per 
lesson, 40 lessons 
a year, 3 years 

>100 

Whole class 


✓ 


✓ 

✓ 

Dramatic Impact 

Walker et al., 

2011 

✓ 


✓ 

✓ 

✓ 



6-7 

Approximately 60 
minutes per lesson, 
40 lessons 

<50 

Whole class 


✓ 


✓ 


Embedded Story Structure 

Faggella et al., 
2007 



✓ 




✓ 

9 

90-120 minutes a 
day, 9 days 

<50 

Small group 






GeoLiteracy 

Hinde et al., 2007 

✓ 



✓ 


✓ 


6 

90 minutes a day, 
6-12 days 

<50 

Whole class 

✓ 

✓ 




Multicomponent social 

studies instruction 

Vaughn et al., 

2009 



✓ 



✓ 


7 

50 minutes a day, 5 
days a week, 9-12 
weeks 

<50 

Whole class 


✓ 

✓ 

✓ 


Promoting Acceleration 
of Comprehension and 
Content through Text 

Vaughn, Swanson, 
et al., 2013 



✓ 



✓ 


8 

50-54 minutes 
per session, 30 

sessions over 6-8 

weeks 

<50 

Whole class 


✓ 

✓ 

✓ 


Read 180 Enterprise 

Kim et al., 2011 

✓ 

✓ 





✓ 

6 

60 minutes a day, 

4 days a week, 23 

weeks 

50-100 

Whole class/ 
Small group 

✓ 

✓ 




Reading Edge 

Chamberlain 

et al., 2007 

✓ 

✓ 



✓ 



8 

50 minutes a day, 9 

months 

>100 

Whole class/ 
Small group 


✓ 


✓ 


Read Now 

Algozzine, 2004 

✓ 

✓ 



✓ 



6-8 

90 minutes a day, 

5 days a week, 10 

weeks 

50-100 

Whole class 


✓ b 


✓ 


Story Impressions 

Denner et al., 

2003 



✓ 



✓ 


8 

One 20-minute 

session 

<50 

Large group 


✓ c 




Student Team Reading and 
Writing 

Stevens, 2003 



✓ 


✓ 

✓ 


6-8 

Daily for school year 

>100 

Whole class 


✓ 


✓ 

✓ 

Tutoring 

Rothman & 

Henderson, 2011 




✓ 



✓ 

7-8 

90 minutes a day, 

2 days a week, 6 

months 

<50 

Small group 


✓ 


✓ 



a. A small group has a student-teacher ratio of less than 4:1; a large group has a student-teacher ratio of at least 12:1. 

b. The study authors specified that implementers were provided with coaching during implementation; therefore, it is assumed that implementers also participated in training before 
implementation. 

c. Teachers were provided with a script to deliver instruction. But ongoing support was not provided to teachers because implementation occurred in one, 20-minute session. 
Source: Authors' summary based on studies reviewed. 




less than 50 hours of instruction, two had 50-100 hours, and three offered more than 100 
hours. 

Four of the programs and practices identified as having positive or potentially positive 
effects used smalhgroup configurations for instruction. Two of them were conducted after 
school, and one was held in the summer. The remaining program incorporated both 
whole-class and smalhgroup configurations within a single class period. 

A majority of the programs and practices identified as having positive or potentially positive effects 
included ongoing support to implementers 

All 12 programs and practices identified as having positive or potentially positive effects 
were implemented by a certified teacher. Three of them specified additional qualifications 
for implementers: two required certification in the program being used, and one selected 
only teachers identified by the district as being effective. For one program identified as 
having a potentially positive effect the certified teacher who implemented the practice was 
also the lead researcher. 

All but one of the programs and practices identified as having positive or potentially 
positive effects included an initial professional development session ranging from 2 to 10 
hours. While the opportunity for professional development varied, ongoing support in 
implementation was common: eight of the programs and practices identified as having pos- 
itive or potentially positive effects received ongoing support. This often included in-class 
support consisting of observation and feedback, regular booster sessions, or consultation 
with school leadership. 

Several common instructional elements were identified by the review team across the 12 
programs and practices identified as having positive or potentially positive effects. They 
are described below and summarized in table 4- These instructional elements were not 
evaluated individually, so it is not possible to isolate the effectiveness of individual instruc- 
tional elements in the current review. 

More than half of the programs and practices identified as having positive or potentially positive 
effects included an instructional routine, and two-thirds also included explicit instruction in reading 
comprehension 

Seven of the 12 programs and practices identified as having positive or potentially posi- 
tive effects incorporated their instruction within a predictable instructional routine (see 
table 4). The activities and focus of content varied, but most instructional routines began 
with teacher-led instruction followed by teacher-guided practice and independent or group 
practice. These programs and practices used routines to organize the activities that could 
be implemented with texts associated with an English language arts classroom or a con- 
tent-area classroom. 

Each routine contained multiple instructional elements. Some instructional elements — 
including explicit instruction in reading comprehension strategies, cooperative learning, 
and writing — were common across multiple programs and practices. 


All 12 programs 
and practices 
identified as 
having positive 
or potentially 
positive effects 
were implemented 
by a certified 
teacher. Three of 
them specified 
additional 
qualifications for 
implementers: 
two required 
certification in 
the program 
being used, and 
one selected only 
teachers identified 
by the district as 
being effective 
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Table 4. Common instructional elements and outcome assessments of the 12 programs and practices identified as having positive and 
potentially positive effects 





Common instructional element 



Outcome assessment 


Intervention 

Citation 

Cooperative 

learning 

Explicit 

instruction: 

reading 

comprehension 

Explicit 

instruction: 

vocabulary 

Feedback 

Fluency building 

Instructional 

routine 

Writing 

Nationally 

normed 

Researcher 

developed 

State 

accountability 

Career Start 3 

Woolley et al., 2013 











Reading 

comprehension 

Dramatic Impact 

Walker etal., 2011 








✓ 



General literacy 

Embedded Story Structure 

Faggella-Luby et al., 2007 



✓ 




✓ 

✓ 


Reading 

comprehension 


GeoLiteracy 

Hinde et al., 2007 



✓ 







Reading 

comprehension 


Multicomponent social 

studies instruction 

Vaughn et al., 2009 

✓ 



✓ 



✓ 

✓ 


Vocabulary 


Promoting Acceleration 
of Comprehension and 
Content through Text 

Vaughn, Swanson, et al., 

2013 

✓ 




✓ 


✓ 


Reading 

comprehension 

Reading 

comprehension 


Read 180 Enterprise 

Kim et al., 2011 



✓ 

✓ 


✓ 

✓ 


Reading 

comprehension 

Vocabulary 



Reading Edge 

Chamberlain et al., 2007 

✓ 


✓ 


✓ 


✓ 


Vocabulary 



Read Now 

Aigozzine, 2004 



✓ 


✓ 

✓ 

✓ 


General literacy 



Story Impressions 

Denner et al., 2003 








✓ 


Reading 

comprehension 


Student Team Reading and 
Writing 

Stevens, 2003 

✓ 


✓ 

✓ 



✓ 

✓ 



Reading 

comprehension 

Vocabulary 

Tutoring 3 

Rothman & Henderson, 2011 











General literacy 


a. These two programs do not include any common instructional elements with the other programs. Career Start provides examples of careers related to content-area topics, and 
Tutoring is an afterschool program focused on practicing and acquiring strategies for taking the state accountability test. 

Source: Authors' summary based on studies reviewed. 




In six of the programs and practices identified as having positive or potentially positive 
effects, teachers taught students reading comprehension strategies and methods to use 
before, during, or after reading to facilitate comprehension. This explicit instruction was 
often repeated, and students were given varied texts and numerous opportunities to apply 
the strategies in whole-class and smalhgroup activities with the teacher, with peers, and 
independently. 

Cooperative learning was implemented in 4 of the 12 programs and practices. All of them 
provided explicit structure and directions to guide student interactions, although the spe- 
cific activities during cooperative learning varied. 

Five of the 12 identified programs and practices integrated writing instruction with reading 
instruction. The writing component was typically used in response to texts the student had 
received and was included in both English language arts classes and content-area classes. 

Studies of the 12 programs and practices identified as having positive or potentially positive effects 
reported small to moderate impacts on state accountability, nationally normed, and researcher- 
developed outcome assessments of reading comprehension, vocabulary, or general literacy 

The programs and practices identified as having positive or potentially positive effects had 
one of three types of outcome assessments: state accountability, nationally normed, and 
researcher developed. Researcher-developed outcome assessments are considered to direct- 
ly reflect the skills targeted by the program or practice. In the 12 programs and practices 
identified as having positive or potentially positive effects, these outcome assessments were 
similar to unit tests commonly used by classroom teachers. None of the researcher-devel- 
oped measures reported here was considered overaligned with the intervention, meaning 
that the assessment items had not been directly taught. 

Eight of the 12 identified studies included state accountability or nationally normed 
outcome assessments (see table 4). The other four included only researcher-developed 
measures. 

The review team followed the WWC procedures for reporting study findings on all types 
of outcome assessments using an effect size and an improvement index. An effect size is a 
standardized measure of the effect of an intervention on student outcome assessments; it 
represents the change (measured in standard deviations) in an average student’s outcome 
that can be expected if that student is given the intervention. The effect size here is 
Hedges’ g. Effect sizes around .20 are considered to be small, .50 to be moderate, and .80 
and greater to be large (Cohen, 1988). The improvement index (II), an alternate presenta- 
tion of the effect size that may be more familiar to educators, reflects the expected change 
in percentile rank of an average student who receives the intervention. 

Four programs and practices included results for the state accountability outcome assess- 
ment, with effect sizes for reading comprehension ranging from g = .21 to .36 (II = 8-14), 
those for vocabulary equal to g = .46 (II = 18), and those for general literacy ranging from 
g = .32 to .40 (II = 12-16). In addition, four programs and practices included results on 
nationally normed outcome assessments, such as assessments of reading comprehension 
(effect sizes ranged from g = .21 to .49; II = 8-15), vocabulary (g = .15 to .46; II = 6-15), 
and general literacy (g = .36; II = 14). 


Among the 
programs and 
practices identified 
as having positive 
or potentially 
positive effects, 
the researcher- 
developed outcome 
assessments 
had the largest 
effect sizes 
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Among the programs and practices identified as having positive or potentially positive 
effects, the researcher-developed outcome assessments had the largest effect sizes, ranging 
fromg = .27 to 1.28 (II = 11-40) for reading comprehension andg = .33 to .50 (II = 12-18) 
for vocabulary (see table Cl in appendix C). Previous research has shown that effect sizes 
tend to be higher on researcher-developed outcome assessments than on assessments that 
measure a more generalized skill set, such as state accountability or nationally normed 
assessments (Swanson, Hoskyn, & Lee, 1999). 

Implications of the review findings 


Of the 33 studies of adolescent literacy programs and practices conducted in the past 20 
years that the review team determined had met WWC standards, only 12 — representing 
12 programs and practices — were identified as having positive or potentially positive effects 
on reading comprehension, vocabulary, or general literacy outcome assessments. Several 
conclusions can be drawn about these 12 programs and practices. Six of the 12 included 
explicit instruction in reading comprehension and seven included instructional routines. 
Nine of the 12 programs and practices were implemented by typical English language arts 
or content-area teachers and often included ongoing support for the implementers. None 
of the 12 programs and practices was conducted in a high school setting. Eight of the 12 
programs and practices were identified as having small to moderate effects on high-stakes 
outcome assessments, such as state accountability and nationally normed assessments. 

The largest increases in student performance were seen for researcher-developed outcome 
assessments, such as a unit test. Although these outcome assessments do not represent 
more generalized skills, they do provide evidence that the implemented program or prac- 
tice improved the targeted skills. 

Statistically significant and substantively important effects also were found on two outcome 
assessments of interest to many educators: state accountability and nationally normed 
assessments. The students participating in the eight studies identified as having potentially 
positive effects on a state accountability or nationally normed assessment improved their 
performance by 6-19 percentile points. Although this is a solid improvement, educators 
should not assume that implementing these programs and practices in different settings 
will result in the same improvements, because these effects may be due to a number of 
other student, environment, or implementation variables (Scammacca et al., 2007). 

This review highlights several common instructional elements, such as explicit instruc- 
tion in comprehension, writing, and cooperative learning, that could be explored in future 
research. Common instructional elements can help inform increasingly effective practic- 
es. Systematic combinations of these elements and organizational components (such as 
extended instruction or professional development) could continue to be investigated to 
determine which elements are necessary and how many are sufficient to demonstrate posi- 
tive effects (Biancarosa & Snow, 2006). 

Adolescent literacy instruction remains an active and important area of inquiry. However, 
additional studies are needed to build the knowledge base on the implementation of 
effective programs and practices for a broader population of adolescents. Only one of the 
studies that were identified as having positive or potentially positive effects included stu- 
dents beyond middle school. And two programs show mixed effects for students in grade 


This review 
highlights 
several common 
instructional 
elements, such as 
explicit instruction 
in comprehension, 
writing, and 
cooperative 
learning, that 
could be explored 
in future research 
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9 (see table C2 in appendix C). Each of these programs was implemented in at least two 
studies in which statistically significant positive effects were found in one study and inde- 
terminate effects were found in the other studies. Additional rigorous research is needed to 
help practitioners with students in grades 9-12. 

Currently available programs and practices could undergo continuing rigorous research to 
expand the understanding of how effective they are. Research is also needed in adolescent 
literacy to expand the number and types of programs and practices available while provid- 
ing scientific evidence of their effectiveness. 

Limitations of this review 


This review used a systematic process for conducting the literature search and study reviews 
to avoid bias. Nonetheless, this review has some limitations. First, the findings are based 
on a small number of studies (in most cases, only one study supported each instructional 
program or practice). This may be due, in part, to the fact that non-peer-reviewed litera- 
ture, such as dissertations and reports from commercial publishers, were excluded from the 
review. Similarly, the common instructional elements discussed in this review were deter- 
mined by the review team and were based on interpretations of the program or practice 
description provided in each of the 12 studies identified as having positive or potentially 
positive effects. Therefore, this review may include instructional elements that may not 
fully capture the instructional elements the study author would have identified. 


Currently available 
programs and 
practices could 
undergo continuing 
rigorous research 
to expand the 
understanding 
of how effective 
they are 


Second, the outcome assessments of interest in the current review are reading compre- 
hension, vocabulary, and general literacy. Discussions of effects on other literacy outcome 
assessments, such as alphabetics or reading fluency, were beyond the scope of this review. 
In addition, this review included only programs and practices that had been studied in 
a general education population. Programs and practices could also be studied exclusively 
among students with learning disabilities or English learner students, and these studies 
might also have had positive effects for general education students. 
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Appendix A. The search, screening, and review process 


This appendix describes the literature search, screening, and review processes used in this 
review. The protocol was adapted from the What Works Clearinghouse (WWC) Adoles- 
cent Literacy Protocol Version 3.0 (http://ies.ed.gov/ncee/wwc/documentsum.aspxlskh29) 
for use by the Regional Educational Laboratory (REL) Southeast. Some key differences in 
protocols as adapted for this review include: 

• Grade range of participants was restricted to students in grades 6-12 (ages 11-18). 

• Participants had to reside in the United States. 

• Eligible outcome domains were restricted to reading comprehension, vocabulary, 
and general literacy. 

• Sample attrition boundary was changed from liberal to conservative. 

This review includes 18 studies previously reviewed by WWC (Alfassi, 1998; Alfassi, 2004; 
Chamberlain et al., 2007; Corrin et al., 2008; Evans-Andris & Usui, 2008; Given et al., 
2008; Graves et al., 2011; Grossen, 2004; Hasselbring & Goin, 2004; Kemple et al, 2008; 
Lang et al., 2009; Nunnery & Ross, 2007; Nunnery, Ross, & McDonald, 2006; Rasinski, 
et al., 2011; Ross et al, 2004; Sandora, Beck, & Mckeown, 1999; Shippen et al, 2005; 
Stevens, 2003). In all cases the review team re-reviewed these studies using the protocol 
outlined below. Four of these studies were determined to have a rating that differed from 
the WWC rating (Corrin et al, 2008; Given et al, 2008; Lang et al., 2009; Nunnery & 
Ross, 2007). Differences in study ratings between this review and the WWC stem from 
differences between the two review protocols. 

The following research question guided the systematic review of evidence outlined below: 

• Among programs and practices studied in the past 20 years that are intended to 
improve reading comprehension, vocabulary, or general literacy, which are effec- 
tive for adolescents in grades 6-12 (ages 11-18)? 

Literature search 

The literature search involved a search of databases (see table Al for keywords), by check- 
ing the references of meta-analyses and literature reviews focused on adolescent literacy 
and by reviewing WWC intervention reports on the subject. This review cast a wide net to 
ensure that the review included all relevant research on adolescent instructional programs 
and practices. Together, these three approaches resulted in the identification of 7,144 
unique reports. 

Six categories of search strings were used to search the following databases (see table Al 
for keywords): 

• ERIC. 

• PsychINFO. 

• EconLit. 

• Child Development and Adolescent Studies. 

The references of 19 meta-analyses and literature reviews were examined and cross- 
referenced with the results of the literature search to identify any additional studies that 
may not have been captured in the search. The references included Berkeley, Scruggs, 
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& Mastropieri, 2010; Bowers, Kirby, & Deacon, 2010; Deshler, Palincsar, Biancarosa, & 
Nair, 2007; Dignath & Buttner, 2008; Edmonds et al, 2009; Elbaum, Vaughn, Elughes, & 
Moody, 2000; Faggella-Luby, Ware, & Capozzoli, 2009; Flynn, Zheng, & Swanson, 2012; 
Guthrie, Schafer, Seeker, & Alban, 2000; Joseph & Schisler, 2009; Kamil et al, 2008; Kim 
& Quinn, 2013; Ma, Adesope, Nesbit, & Liu, 2014; Pearson, Ferdig, Blomeyer, & Moran, 
2005; Scammacca et al., 2007; Slavin, Cheung, Groff, & Lake, 2008; Strong, Torgerson, 
Torgerson, & Hulme, 2011; Suggate, 2010; Wanzek, Wexler, Vaughn, & Ciullo, 2010, and 
24 WWC intervention reports (U.S. Department of Education, 2009a, 2009b, 2010a, 
2010b, 2010c, 2010d, 2010e, 2010f, 2010g, 2010h, 2010i, 2010j, 2010k, 2011a, 2011b, 2012a, 


Table Al. Keywords used in database search 

Topic 

Curricula and 
practice names 

Intervention 

Evaluation 

Literacy 

Corrective Reading 

Intervention* 

Control group* 

Reading development 

Directed reading activity 

Curricul* 

Random* 

Reading 

Accelerated Reader 

Program* 

Simultaneous treatment 

Critical literacy 

Book Club 

Strateg* 

Comparison group* 

Functional literacy 

Fast Forward 

Instruct* 

Regression discontinuity 

Literacy 

Great Books 

Teach* 

Matched group* 

Information literacy 

High School Puente 
Program 

Train* 

Baseline 

Literacy education 

Language! 

Educational therapy 

ABAB design 

Scientific literacy 

Open Court Reading 

Practice 

Treatment 

Critical reading 

Peer-Assisted Learning 
Strategies 

Approach 

Experiment 


Grades 6-12 Project CRISS (Creating Technique* Meta-analysis 


Independence through 

Student-oriented 

Strategies) 


Grade 6 

READ 180 

Literacy instruction 

Meta-analysis 

Grade 7 

Read Naturally 

Basal reading 

Evaluation 

Grade 8 

Reading Apprenticeship 

Remedial reading 

Assignment 

Grade 9 

Reading Edge 

Reading instruction 

Impact 

Grade 10 

Reading Plus 

Literacy program* 

Effectiveness 

Grade 11 

Reading Mastery 

Reading education 

Causal 

Grade 12 

Reciprocal Teaching 

Literacy education 

Post-test 

Middle school student 

The Spaulding Method 

Educational strateg* 

Pretest 

High school student 

SpellRead 

Educational method* 

Randomized Control Trial 

Secondary school 

student 

Student Team Reading 
and Writing 

Instructional design 

RCT 

Adolescent* 

SuccessMaker 

Learning strategies 

Quasi-experimental 

design 

Adolescence 

Talent Development 
Middle Grades Program 

Instructional strategies 

QED 


ReadAbout 

Instructional materials 

Changing criterion design 


Read for Real 

Courseware 

Intra-subject replication 
design 


Reading for Knowledge 

Learning modules 

Multiple baseline design 

( continued) 
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Table Al. Keywords used in database search (continued) 

Curricula and 


Topic practice names Intervention 

Evaluation 

Textbooks 

Multi-element design 

Workbooks 

Multi-element design 

Protocol materials 

Single-case design 

Reading materials 

Single-subject design 

Educational games 

Alternating treatment 

Educational resources 

Reversal design 

Material development 

Withdrawal design 

Instructional media 

Post-test 

Instructional 

Pretest 

effectiveness 


Instructional 

Quasi-experimental 

improvement 

design 

Program effectiveness 

Administrator 


effectiveness 


Curriculum evaluation 

Educational quality 

Outcomes of education 

Assignments 

Homework 

Reading assignments 

Schoolwork 

* is a wildcard symbol used in the literature search engine to retrieve variations of the word stem. 

Source: Authors' compilation. 



2012b, 2012c, 2012d, 2012e, 2013a, 2013b, 2013c, and 2013d). This process resulted in the 
identification of some 150 additional studies. 

Screening process and study eligibility criteria 

The abstracts and full reports of the 7,144 studies identified through the literature search 
were screened to determine whether they were eligible for review. To be eligible, studies 
had to meet the following relevancy criteria: 

• Topic. The study had to be about adolescent literacy and focused on the effects of 
interventions and not on individual differences (such as correlational studies) or 
assessment. Following were the types of interventions that could be included: 

o Programs/products (such as comprehensive non-textbook-based programs, sup- 
plemental programs, programs aimed at struggling readers, or software). 

° Practices (such as vocabulary instruction, questioning, or summarizing), 
o Policies (such as a schoolwide literacy initiative), 
o Variations across programs, products, practices, and policies, 
o Commercial programs. 

• Time. The study had to be publicly released between 1994 and 2014. 

• Publication status. The study had to be published in a journal or the work sup- 
ported by federal funds that required the study to be peer-reviewed by external 
reviewers. 
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• Sample. The study needed to satisfy three sample-specific criteria: 

o Grade levels. The study had to address outcomes of students in grades 6-12 
(ages 11-18). If a study included a sample of students spanning grades 6-12 
and K-5 and the reported results could not be disaggregated by grade level, 
the study would still be reviewed as long as it included grade 7 students or 
higher. And when study authors included a longitudinal sample of students 
who received the intervention in grades 6-12 or K-5, any studies in which 
the students received the intervention in grade 7 or higher were eligible for 
review. 

o Achievement level. At least 50 percent of the students in each study had to be 
general education students and not English learner students, 
o Location of the intervention. The students in the sample had to reside in the 
United States or in one of its territories. 

• Outcomes. The study needed to include at least one student-level outcome in any 
of the following domains: 

o Reading comprehension. Reading comprehension depends on various under- 
lying components, including knowledge of word meanings along with the 
ability to translate text into speech (decoding), to read text accurately and 
automatically (fluency), and to understand and interpret the spoken language. 
Struggling readers may have difficulty with one or more of these components 
of reading. Reading comprehension outcomes may include tests of students’ 
comprehension of text from various content areas — such as a social studies 
passage. 3 State outcome assessments or standardized assessments are included 
as a measure of reading comprehension as long as the score is based solely on 
students answering questions about text. 

o Vocabulary. Critical to improving reading comprehension is building knowl- 
edge of the meanings and uses of words. This includes developing both recep- 
tive vocabulary (words understood) and expressive vocabulary (words used), 
o General literacy. Outcomes here incorporate measurement of two or more of 
any reading-related domains by providing some type of summary score, such as 
a “total reading score” on a standardized reading test across domains. Grades 
in reading, English, or language arts classes are not acceptable outcomes. If 
reading comprehension is one component of the task, along with tasks that 
require separate reading or writing activities, state outcome assessments or 
standardized assessments will be included as a measure of general literacy. 

• Study design. Studies that used a randomized controlled trial, quasi-experimental 
design, or a single-case design were included. 

This process screened out 7,033 studies, leaving 111 to be reviewed by a WWC-certified 
reviewer. 4 

Reviewing studies using What Works Clearinghouse evidence standards 

Studies were reviewed using the WWC Procedures and Standards Handbook (version 3.0) for 
group design and single-case design studies (U.S. Department of Education, 2014). Studies 
that the first reviewer determined had met standards were independently reviewed by a 
second reviewer. A senior reviewer then double-checked the first and second reviews to 
ensure accuracy and reconcile any differences between the two. The summary of programs 
and practices described in this report includes only instructional programs and practices 
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that two reviewers determined had met WWC evidence standards (version 3.0) with or 
without reservations. 

Of the 111 studies reviewed, 78 were rated by the review team as not meeting the WWC 
evidence standards; table A2 lists the primary reasons. Of these, 60 percent were not able 
to establish baseline equivalence for the treatment and comparison groups used for the 
statistical analysis, 21 percent included a confound, and 15 percent did not include a valid 
and reliable outcome. A confound can occur when only one unit is assigned to one or all 
conditions or the intervention was always used in combination with another intervention. 
The presence of a confound means that the study findings cannot be attributed solely to 
the intervention. 

The remaining 33 studies that the review team identified as having met WWC standards 
represented 29 different interventions and classroom practices. Of these interventions, 1 
was identified as having a positive effect, 11 as having potentially positive effects, 2 as 
having mixed effects, 15 as having no discernible effects, and none as having potentially 
negative or negative effects. 


Table A2. Reasons for not meeting What Works Clearinghouse evidence standards 
rating, as determined by the review team 

Primary reason for the rating indicating a failure 

Number of 
studies with 

Share of total 

to meet WWC evidence standards 

rating 

(percent) 

The outcome assessments were not shown to be valid and reliable 

12 

15 

There was only one unit assigned to one or both conditions (confound) 

16 

21 

Baseline equivalence was not demonstrated 3 

47 

60 

No graphical representation of evidence was provided 11 

1 

1 

There were insufficient data points 3 

2 

3 

Total 

78 

100 


WWC is What Works Clearinghouse. 

Note: Percentages may not sum to 100 because of rounding. 

a. According to WWC standards for group design, baseline equivalence should be demonstrated for quasi-ex- 
perimental design studies as well as for randomized controlled trials with high attrition based on a conserva- 
tive boundary or randomization problems. 

b. According WWC pilot standards for single-case design studies, graphical representation of data points, and 
a sufficient number of data points are required to receive a rating of meeting standards. 

Source: Authors' compilation. 




Appendix B. The 111 studies reviewed 
using What Works Clearinghouse standards 


Of the 7,144 studies identified in the literature search, the 111 studies included in this 
appendix used a research design for examining the effectiveness of interventions on 
reading comprehension, vocabulary, or general literacy for students in grades 6-12 residing 
in the United States. An asterisk indicates the 33 of those 111 studies that the review 
team determined met What Works Clearinghouse evidence standards (version 3.0) with or 
without reservations. A dagger indicates the 18 of those 111 studies that had been previ- 
ously reviewed by the What Works Clearinghouse using the Adolescent Literacy Protocol 
Version 3.0 and were re-reviewed by the review team using the modified protocol described 
in appendix A. 

*Alber, S. R., Nelson, J. S., & Brennan, K. B. (2002). A comparative analysis of two homework 
study methods on elementary and secondary school students’ acquisition and mainte- 
nance of social studies content. Education and Treatment of Children, 25(2), 172-196. 
http://eric.ed.gov/lkUEJ655474 

Udfassi, M. (1998). Reading for Meaning: The efficacy of reciprocal teaching in fostering 
reading comprehension in high school students in remedial reading classes. American 
Educational Research Journal, 35(2), 309-332. http://eric.ed.gov/lkUEJ574570 

LMfassi, M. (2004). Reading to learn: Effects of combined strategy instruction on high 
school students. The Journal of Educational Research, 97(4), 171-184. http://eric.ed.gov/ 
?id=EJ698471 

*Algozzine, B. (2004). Effects of Read Now on adolescents at risk for school failure. Journal 
of At-Risk Issues, 10(2), 1-8. 

*Allinder, R., Dunse, L., Brunken, C. D., & Obermiller-Krolikowski, H. J. (2001). Improv- 
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Special Education, 22(1), 48-54. http://eric.ed.gov/lkUEJ623056 

Anderson, S., & Mezuk, B. (2012). Participating in a policy debate program and academic 
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Journal of Adolescence, 35(5), 1225-1235. http://eric.ed.gov/lkUEJ989157 

Backes, J., Ralston, A., & Ingwalson, G. (1999). Middle level reform: The impact on 
student achievement. Research in Middle Level Education Quarterly, 22(3), 43-57. http:// 
eric.ed.gov/?id=EJ584445 

Balfanz, R., Legters, N., & Jordan, W. (2004). Catching up: Effect of the talent development 
ninth-grade instructional interventions in reading and mathematics in high-poverty 
schools. National Association of Secondary School Principals NASSP Bulletin, 88(641), 
3-30. http://eric.ed.gov/lkUEJ747957 

*Belcher, C., Frey, A., & Yankeelov, P. (2006). The effects of single-sex classrooms on 
classroom environment, self-esteem, and standardized test scores. School Social Work 
Journal, 31(1), 61-75. 
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Biggs, M. C., Homan, S. P., Dedrick, R., Minick, V., & Rasinsky, T. (2008). Using an inter- 
active singing software program: A comparative study of struggling middle school 
readers. Reading Psychology, 29(3), 129-213. http://eric.ed.gov/lkUEJ799231 

Bogner, R. G., Salvadore, F., & Manley, E. (2008). The effects of peer mediation programs 
on middle school academic performance. Perspectives in Peer Programs, 21(2), 59-68. 
http://eric.ed.gov/lkUEJ846148 

*Borman, G. D., Benson, J. G., & Overman, L. (2009). A randomized field trial of the 
Fast ForWord Language computer-based training program. Educational Evaluation and 
Policy Analysis, 31(1), 82-106. http://eric.ed.gov/lkUEJ869816 

Braun, H., Kirsch, I., & Yamamoto, K. (2011). An experimental study of the effects of 
monetary incentives on performance on the 12th-grade NAEP reading assessment. 
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Brigman, G., & Campbell, C. (2003). Helping students improve academic achievement 
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impact of a strategy based intervention on the comprehension and strategy use of 
struggling adolescent readers. Journal of Educational Psychology, 102(2), 257-280. http:// 
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Appendix C. Details of the 33 studies that the review team 
found had met What Works Clearinghouse evidence standards 


This appendix reports the research design, analysis sample, intervention implementation, 
nature of the comparison group, student outcome assessments, effect size, and improve' 
ment index for the 33 studies that the review team determined had met What Works 
Clearinghouse evidence standards with or without reservations. Tables are organized based 
on whether the study showed positive or potentially positive effects (table Cl), mixed 
effects (table C2), or indeterminate effects (table C3). 
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Table Cl. Adolescent literacy programs and practices identified by the review team as having positive and potentially positive effects 

Full citation 

Program or 
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Research 

design 

Analysis sample 

Intervention 

implementation 

Nature of the 
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Student outcome 
assessment 

Effect size 3 
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Improvement 
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Effects of Read Now on 

adolescents at risk for 
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Quasi-experimental 
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schools in four 

states 
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of Read Now daily 
for 10 weeks 
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remedial reading 
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(n = 116) 

STAR Reading 
(assessment of 
comprehension, 
vocabulary, and 
fluency; nationally 
normed) 

.36* 

+14 

Chamberlain, A., Daniels, 
C., Madden, N. A., & 

Reading Edge 

Randomized 

controlled trial 

405 grade 6 
students in two 

Reading Edge, 
implemented for 

Schools' typical 
instruction 

GMRT Vocabulary 
(nationally normed) 

.15* 

+6 

Slavin, R. E. (2007). A 
randomized evaluation of 

the Success for All middle 
school reading program. 



rural high-poverty 
schools in West 
Virginia and Florida 

60 minutes daily 
for a full year 
(n = 203) 

(n = 202) 

GMRT Reading 
Comprehension 
(nationally normed) 

,ll c 

+4 


Middle Grades Research 
Journal, 2(1), 1-21. 


Denner, P. R., Rickards, 

Story 

Randomized 

74 grade 8 

Story Impressions 

Content preview 

A historical 

1.28* 

+40 

J. P., & Albanese, A. J. 

Impressions 

controlled trial 

students 

(n = 24) 

(n = 25) 

biography passage 



(2003). The effect of Story 



assigned to Story 



with 45 fill-in- 



Impressions preview on 



Impressions 



the-blank facts 



learning from narrative 



group, content 



from the passage 



text. The Journal of 



preview group, or 



(researcher 



Experimental Education, 
71(4), 313-332. 



control group d 



developed) 



Faggella-Luby, M., 

Embedded Story 

Randomized 

79 at-risk students 

Embedded Story 

Comprehension 

A unit reading 

.83* 

+30 

Schumaker, J. S., & 

Structure Routine 

controlled trial 

entering grade 

Structure (n = 39) 

skill instruction 

comprehension 



Deshler, D. D. (2007). 



9, including 


o 

•si- 

ll 

test with 40 



Embedded learning 



14 students 



short- answer and 



strategy instruction: Story- 



with learning 



fill-in-the-blank 



structure pedagogy in 



disabilities 



questions related 



heterogeneous secondary 






to the eight stories 



literature classes. 






read by both 



Learning Disability 






groups (researcher 



Quarterly, 30(2), 131-147. 






developed) 



Hinde, E. R., Popp, 

GeoLiteracy 

Quasi-experimental 

487 grade 6 

GeoLiteracy 

Teachers’ 

A reading 

.27* 

+11 

S. E. 0., Dorn, R. 1., 



students 

lessons 

standard social 

comprehension 



Ekiss, G. 0., Mater, 




(n = 317) 

studies curriculum 

test with 10 



M., Smith, C. B., et al. 





(n = 170) 

multiple-choice 



(2007). The integration of 
literacy and geography: 

The Arizona GeoLiteracy 
program's effect on 
reading comprehension. 
Theory & Research in 

Social Education, 35(3), 
343-365. 



247 grade 8 
students 

GeoLiteracy 
lessons 
(n = 164) 

Teachers’ 

standard social 

studies curriculum 
( n = 83) 

questions about 
text (researcher 
developed) 

.38* 

+15 


(continued) 
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Table Cl. Adolescent literacy programs and practices identified by the review team as having positive and potentially positive effects (continued) 


Program or 

Research 


Intervention 

Nature of the 

Student outcome 

Effect size 3 

Improvement 

Full citation 

practice 

design 

Analysis sample 

implementation 

comparison group 

assessment 

(Hedges’ g) 

index b 

Kim, J. S., Capotosto, L.. 

Read 180 

Randomized 

56 grade 6 

Read 180 

Teacher-created 

SAT-10 Reading 

.491' 

+19 

Harty, A., & Fitzgerald, 

Enterprise 

controlled trial 

students who 

Enterprise 

reading instruction 

Comprehension 



R. (2011). Can a 



failed to meet 

(n = 29) 

in the same after- 

(nationally normed) 



mixed-method literacy 
intervention improve the 



standards on the 

Massachusetts 


school program 
location 

SAT-10 Vocabulary 
(nationally normed) 

.40 f 

+15 

reading achievement of 



Comprehensive 


(n = 27) 



low-performing elementary 



Assessment 






school students in an 



System in English 






after-school program? 
Results from a 

randomized controlled trial 
of READ 180 Enterprise. 
Educational Evaluation 
and Policy Analysis, 33(2), 
183-201. 



language arts 






Rothman, T., & 

Tutoring 

Quasi-experimental 

60 grade 8 

Tutoring using 

Participants 

Proficiency on 

.40* 

+16 

Henderson, M. (2011). 



students from 

Preparing for the 

received the 

NJASK English 



Do school-based tutoring 



one school that 

New Jersey Grade 

same small-group 

language arts 



programs significantly 



were designated 

Eight Proficiency 

tutoring for the 

subtest (state 



improve student 



as “borderline” 

Assessment 

same duration as 

accountability) 



performance on 



based on their 

and Standard 

in the intervention 




standardized tests? 



grade 7 scores 

Solutions ( n = 23) 

group but focused 




Research in Middle Level 



on the state 


on math (n = 37) 




Education Online, 34(6), 



accountability 






1-10. 



assessment 






Stevens, R. J. (2003). 

Student Team 

Quasi-experimental 

3,916 grade 6-8 

Student Team 

Typical classroom 

CST reading 

,36t 

+14 

Student team reading 

Reading and 


students in five 

Reading and 

instruction 

comprehension 



and writing. A cooperative 

Writing 


large urban middle 

Writing (n = 1,798 

(n = 2,188 

subtest (state 



learning approach to 



schools 

students in 72 

students in 88 

accountability) 



middle school literacy 
instruction. Educational 




classes in two 
schools) 

classes in three 
schools) 

CST vocabulary 
subtest (state 

,46f 

+18 

Research and Evaluation, 
9(2), 137-160. 






accountability) 



Vaughn, S., Martinez, 

Multicomponent 

Cluster randomized 

Study 1: 381 

Multicomponent 

Typical classroom 

Social studies 

.50* 

+18 

L. R., Linan-Thompson, 

social studies 

controlled trial 

grade 7 students 

social studies 

instruction (n = 8 

unit vocabulary 



S., Reutebuch, C. K., 

instruction 


in 15 classrooms 

instruction (n = 7 

classrooms) 

test (researcher 



Carlson, C. D., & Francis, 



with 25 percent 

classrooms) 


developed) 



D. J. (2009). Enhancing 



English learners 






social studies vocabulary 
and comprehension for 
seventh-grade English 
language learners: 

Findings from two 
experimental studies. 
Journal of Research on 
Educational Effectiveness, 
2(4), 297-324. 



Study 2: 507 
grade 7 students 
in 17 classrooms 
with 21 percent 
English learners 

Multicomponent 
social studies 
instruction (n = 8 
classrooms) 

Typical classroom 
instruction (n = 9 
classrooms) 


.33* 

+12 


(continued) 
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Table Cl. Adolescent literacy programs and practices identified by the review team as having positive and potentially positive effects (continued) 

Full citation 

Program or 
practice 

Research 

design 

Analysis sample 

Intervention 

implementation 

Nature of the 
comparison group 

Student outcome 
assessment 

Effect size 3 
(Hedges' g) 

Improvement 

index b 

Vaughn, S., Swanson, E. 

A., Roberts, G., Wanzek, 

J., Stillman-Spisak, 

Promoting 
Acceleration of 
Comprehension 

Randomized 

controlled trial 

344 grade 8 
students from 27 

classes in two 

Promoting 
Acceleration of 
Comprehension 

Typical classroom 

instruction 

(n = 199 students 

GMRT 

Comprehension 
(nationally normed) 

.21* 

+8 

S. J., Solis, M., et al. 
(2013). Improving reading 
comprehension and social 
studies knowledge in 
middle school. Reading 
Research Quarterly, 48(1), 
77-93. 

and Content 
Through Text 


schools 

and Content 
Through Text 
( n = 203 students 
in 16 classes) 

in 11 classes) 

Assessment of 

Social Studies 
Knowledge 

Reading 

comprehension 

subtest 

(researcher 

developed) 

.35* 

+14 

Walker, E., Tabone, C., 

& Weltsek, G. (2011). 

When achievement data 

meet drama and arts 
integration. Language 

Arts, 88(5), 365-372. 

Dramatic Impact 

Randomized 

controlled trial 

699 grade 6 and 7 
students from 28 

classrooms and 8 

schools 

Dramatic Impact 
implemented in 
English language 
arts classrooms 

over the course of 
the academic year 
(n = 390) 

Teachers’ 
standard English 
language arts 
curriculum using 
the same literary 
texts (n = 309) 

Proficiency on 

NJ ASK English 
language arts 
subtest (state 
accountability) 

,32t 

+12 

Woolley, M. E., Rose, R. 

A., Orthner, D. K., Akos, 

P. T., & Jones-Sanpei, 

H. (2013). Advancing 
academic achievement 
through career relevance 
in the middle grades: A 
longitudinal evaluation 
of CareerStart. American 

Educational Research 
Journal, 50(6), 1309- 
1335. 

CareerStart 

Cluster randomized 

controlled trial with 
schools assigned 
to condition 

3,295 students 
from 14 schools 

were followed from 
grade 6 through 
grade 8 

Students 
participated in 
CareerStart (n = 7 
schools) 

Typical 
instructional 
practices (n = 1 
schools) 

North Carolina 
end-of-grade 
reading test (state 
accountability) 

,46 + 

+18 


* Effect size was statistically significant at p < .05. 

f Effect size was substantively important; effect sizes of .25 or greater are considered substantively important, regardless of the statistical significance, according to What Works 
Clearinghouse (WWC) evidence standards (version 3.0). 

GMRT is Gates McGinitie Reading Test; SAT-10 is Stanford Achievement Test, Tenth Edition; NJASK is New Jersey Assessment of Skills and Knowledge; CST is California Standards Test. 

a. A standardized measure of the effect of an intervention on student outcomes representing the change (measured in standard deviations) in an average student's outcome that can be 
expected if that student is given the intervention. 

b. An alternate presentation of the effect size, reflecting the change in an average student’s percentile rank that can be expected if the student is given the intervention. 

c. This effect is neither statistically significant nor substantively important. 

d. Only the comparison between the two intervention groups met WWC group design standards. The comparison to the control group did not meet standards. 

Source: Authors' summary of studies. 
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Table C2. Adolescent literacy programs and practices identified by the review team as having mixed effects 

Full citation 

Program or 
practice 

Research design 

Analysis sample 

Intervention 

implementation 

Nature of the 
comparison group 

Student outcome 
assessments 

Effect size 3 
(Hedges’ g) 

Improvement 

index b 

Corrin, W., Lindsay, 

J. J., Somers, M. A., 

Myers, N. E., Meyers, 

Content Literacy 
Continuum 

Cluster randomized 

controlled trial 

28 schools of 
grade 9 students, 
cohort 1 

Content Literacy 
Continuum (n = 15 
schools) 

Typical 
instructional 
practices (n = 13 

GRADE Reading 
Comprehension 
(nationally normed) 

.13 

+5 

V., Condon, C. A., et al. 
(2012). Evaluation of the 
content literacy continuum: 
Report on program 
impacts, program fidelity, 
and contrast (NCEE No. 





schools) 

GRADE Vocabulary 
(nationally normed) 

.13* 

+5 



33 schools of 
grade 9 students, 
cohort 2 

Content Literacy 
Continuum ( n = 17 
schools) 

Typical 
instructional 
practices n = 16 

GRADE Reading 
Comprehension 
(nationally normed) 

.06 

+2 

2013-4001). Washington, 
DC: U.S. Department 





schools) 

GRADE Vocabulary 
(nationally normed) 

.09 

+4 

of Education, Institute 
of Education Sciences, 
National Center for 

Education Evaluation and 
Regional Assistance. 



28 schools of 
grade 10 students, 
cohort 1 



GRADE Reading 
Comprehension 
(nationally normed) 


+4 






GRADE Vocabulary 
(nationally normed) 

.10 

+4 

Corrin, W., Somers, M. 

A., Kemple, J. J., Nelson, 

E., & Sepanik, S. (2008). 

Reading 

Apprenticeship 

Randomized 

controlled trial 

2,171 grade 9 
students from 

34 high schools 

Reading 
Apprenticeship 
elective or Xtreme 

Other elective 
course (n = 907) 

GRADE Reading 
Comprehension 
(nationally normed) 

.08* 

+3 

The enhanced reading 
opportunities study: 

Findings from the second 



performing at least 
two years below 
grade level 

Reading elective 
(n = 1,264) 


GRADE Vocabulary 
(nationally normed) 

0 

0 

year of implementation 
(NCEE No. 2009-4036). 
Washington, DC: U.S. 
Department of Education, 
Institute of Education 
Sciences, National Center 
for Education Evaluation 
and Regional Assistance. 



1,115 grade 9 
students from 

34 high schools 

Reading 
Apprenticeship 
(n = 645) 

Other elective 
course (n = 470) 

GRADE Reading 
Comprehension 
(nationally normed) 

.14* 

+5 



performing at least 
two years below 
grade level 



GRADE Vocabulary 
(nationally normed) 

-.04 

-2 



1,056 grade 9 
students from 

34 high schools 

Xtreme Reading 
(n = 619) 

Other elective 
course (n = 437) 

GRADE Reading 
Comprehension 
(nationally normed) 

.02 

+1 




performing at least 
two years below 
grade level 



GRADE Vocabulary 
(nationally normed) 

.04 

+2 


(continued) 
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Table C2. Adolescent literacy programs and practices identified by the review team as having mixed effects (continued) 

Full citation 

Program or 
practice 

Research design 

Analysis sample 

Intervention 

implementation 

Nature of the 
comparison group 

Student outcome 
assessments 

Effect size a 
(Hedges’ g) 

Improvement 

index b 

Greenleaf, C., Hanson, 

T., Herman, J., Litman, 

C., Rosen, R., Schneider, 

S., etal. (2011). A 

Reading 

Apprenticeship 

Cluster randomized 

controlled trial 

668 grade 9 and 

10 students in 
biology classes 
in 33 high 

Reading 

Apprenticeship 

professional 

development 

Biology teachers 
were provided 
with curricular 

materials but 

CST English 
language arts 
subtest (state 
accountability) 

0 

0 

study of the efficacy of 
Reading Apprenticeship 
professional development 
for high school history 
and science teaching and 
learning. Washington, 

DC: U.S. Department 
of Education, Institute 
of Education Sciences, 
National Center for 

Education Research. 



schools serving 
underrepresented 
students in 

California 

provided to biology 
teachers (n = 13 
teachers) 

no professional 
development 
(n = 27 teachers) 

CST reading 
comprehension 
subtest (state 
accountability) 

.14 

+6 






DRP 

Comprehension 
(nationally normed) 

-.03 

-1 

Greenleaf, C. L., Litman, 

C., Hanson, T. L., 

Rosen, R., Boscardin, 

C. K., Herman, et al. 

Reading 

Apprenticeship 

Cluster randomized 

controlled trial with 
schools assigned 
to condition 

45 schools with 

54 teachers 
serving 1,111 
grade 9 and 10 

Reading 
Apprenticeship 
(n = 23 schools) 

Typical 
instructional 
practices (n = 22 
schools) 

CST reading 
comprehension 
subtest (state 
accountability) 

.04 

+2 

(2011). Integrating 
literacy and science in 
biology: Teaching and 
learning impacts of 
reading apprenticeship 
professional development. 
American Educational 
Research Journal, 48(3), 
647-717. 



students 



CST English 
language arts 
subtest (state 
accountability) 

.03 

+1 


(continued) 




Table C2. Adolescent literacy programs and practices identified by the review team as having mixed effects (continued) 



Program or 



Intervention 

Nature of the 

Student outcome 

Effect size 3 

Improvement 

Full citation 

practice 

Research design 

Analysis sample 

implementation 

comparison group 

assessments 

(Hedges’ g) 

index b 

Kemple, J. J., Corrin, 

Reading 

Randomized 

2,413 grade 9 

Reading 

Other elective 

GRADE Reading 

.09* 

+4 

W., Nelson, E., Salinger, 

Apprenticeship 

controlled trial 

students from 

Apprenticeship 

course (n = 1,005) 

Comprehension 



T., Herrmann. S., 



34 high schools 

elective or Xtreme 


(nationally normed) 



Drummond, K., et al. 
(2008). The enhanced 
reading opportunities 



performing at least 
two years below 
grade level 

Reading elective 
(n = 1,408) 


GRADE Vocabulary 
(nationally normed) 

.03 

+1 

study: Early impact and 
implementation findings 
(NCEE No. 2008-4015). 
Washington, DC: U.S. 
Department of Education, 
Institute of Education 



1,140 grade 9 
students from 

34 high schools 

Reading 
Apprenticeship 
(n = 686) 

Other elective 
course (n = 454) 

GRADE Reading 
Comprehension 
(nationally normed) 

.09 

+4 



performing at least 
two years below 
grade level 



GRADE Vocabulary 
(nationally normed) 

.05 

+2 

Sciences, National Center 
for Education Evaluation 
and Regional Assistance. 



1,273 grade 9 
students from 

34 high schools 

Xtreme Reading 
(n = 722) 

Other elective 
course (n = 551) 

GRADE Reading 
Comprehension 
(nationally normed) 

.08 

+3 




performing at least 
two years below 
grade level 



GRADE Vocabulary 
(nationally normed) 

.01 

0 


* Effect size was statistically significant at p < .05. 

GRADE is Group Reading Assessment and Diagnostic Evaluation; CST is California Standards Test; DRP is Degrees of Reading Power. 

a. A standardized measure of the effect of an intervention on student outcomes representing the change (measured in standard deviations) in an average student’s outcome that can be 
expected if that student is given the intervention. 

b. An alternate presentation of the effect size, reflecting the change in an average student's percentile rank that can be expected if the student is given the intervention. 

Source: Authors' summary of studies. 
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Table C3. Adolescent literacy programs and practices identified by the review team as having indeterminate effects 

Full citation 

Program or 
practice 

Research 

design 

Analysis sample 

Intervention 

implementation 

Nature of the 
comparison group 

Student outcome 
assessment 

Effect size 3 
(Hedges’ g) 

Improvement 

index b 

Alber, S. R., Nelson, J. S., 

& Brennan, K. B. (2002). 

A comparative analysis 
of two homework study 
methods on elementary 
and secondary school 
students’ acquisition and 
maintenance of social 

studies content. Education 

and Treatment of Children, 
25(2), 172-196. 

Structured 

homework 

assignment 

Single-case 
design: reversal/ 

withdrawal 

12 grade 9 
students in 

general and 
special education 0 

Standard review 
questions study 

method and 

structured reading 

worksheet method 

for homework 

(n = 1 student 
with a learning 
disability) 

Typical homework 
assignment (same 
student) 

Quiz of reading 
comprehension 
(researcher 
developed) 

No 

evidence 01 


Allinder, R., Dunse, 

L., Brunken, C. D., & 
Obermiller-Krolikowski, 

H. J. (2001). Improving 
fluency in at-risk readers 
and students with learning 

disabilities. Remedial and 

Special Education, 22(1), 

48-54. 

Fluency building 

Randomized 

controlled trial 

49 grade 7 

students in three 

classes who were 

considered to be 
struggling readers 

Students practiced 

and were told 

to use an oral 

reading fluency 
strategy (n = 33) 

Students were 

instructed to "do 

your best” (n = 16) 

WRMT-R Passage 
Comprehension 
(nationally normed) 

-.02 

-1 

Belcher, C., Frey, A., & 
Yankeelov, P. (2006). 

The effects of single-sex 
classrooms on classroom 

environment, self-esteem, 

and standardized test 

scores. School Social Work 

Journal, 31(1), 61-75. 

Single-sex 

classrooms 

Randomized 

controlled trial 

98 grade 6 

students in 

general education 

Classrooms 

consisted of 

all males or all 
females ( n = 48) 

Co-educational 

classrooms 

(n = 50) 

Commonwealth 

Accountability 
Testing System- 
Comprehension 
(state 

accountability) 

-.07 

-3 

Borman, G. D., Benson, 

J. G., & Overman, L. 

(2009). A randomized field 
trial of the Fast ForWord 

Language computer- 
based training program. 

Educational Evaluation 

and Policy Analysis, 31(1), 

82-106. 

Fast ForWord 

Language 

Randomized 

controlled trial 

201 grade 7 

at-risk students in 

three urban middle 

schools 

Fast ForWord 

Language 
computer program 
implemented in a 
pullout classroom 
( n = 98) 

Students 

participated 
in nonliteracy 
classes (n = 103) 

CTBS/5 Reading 
Comprehension 
(nationally normed) 

.01 

0 

(continued) 
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Table C3. Adolescent literacy programs and practices identified by the review team as having indeterminate effects (continued) 


Program or 

Research 


Intervention 

Nature of the 

Student outcome 

Effect size 3 

Improvement 

Full citation 

practice 

design 

Analysis sample 

implementation 

comparison group 

assessment 

(Hedges’ g) 

index b 

Drummond, K., Chinen, 

Thinking Reader 

Cluster randomized 

2,156 grade 6 

Thinking Reader 

Typical classroom 

GMRT 

.03 

+1 

M., Duncan, T. G., Miller, 


controlled trial 

students in 90 

computer 

practices (n = 42 

Comprehension 



H. R., Fryer, L., Zmach, C., 



language arts 

program (n = 48 

classrooms) 

(nationally normed) 



et al. (2011). Impact of the 



classrooms 

classrooms) 


GMRT Vocabulary 

-.04 

-2 

Thinking Reader software 
program on grade 6 
reading vocabulary, 
comprehension, strategies, 
and motivation (NCEE 
2010-4035). Washington, 
DC: U.S. Department 
of Education, Institute 
of Education Sciences, 

National Center for 

Education Evaluation and 

Regional Assistance. 






(nationally normed) 



Field, G. B. (2010). From 

Renzulli Learning 

Cluster randomized 

383 students in 

Renzulli Learning 

Typical classroom 

ITBS Reading 

-.06 

-2 

quantitative results to 


controlled trial 

23 classrooms 

online educational 

practices (n = 12 

Comprehension 



qualitative meaning: A 



in one middle 

learning system 

classrooms) 

(nationally normed) 



look at Renzulli Learning 



school and one 

and profiler 





through the eyes of 



elementary school 

implemented 





students, teachers, 



(grades 3-8) 

in Technology 





and administrators at 




Connections 





Oak Hills Elementary 




course (n = 11 





School and Inman Middle 

School. Gifted Education 

International, 26(2-3), 
285-301. 




classrooms) 





(continued) 
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Table C3. Adolescent literacy programs and practices identified by the review team as having indeterminate effects (continued) 

Full citation 

Program or 
practice 

Research 

design 

Analysis sample 

Intervention 

implementation 

Nature of the 
comparison group 

Student outcome 
assessment 

Effect size a 
(Hedges’ g) 

Improvement 

index b 

Fogarty, M., Oslund, 

E., Simmons, D., 

Davis, J . , Simmons, 

Comprehension 
circuit training 

Randomized 

controlled trial 

736 grade 6-8 

students in 61 

classrooms 

Comprehension 
Circuit Training 
(n = 411 students) 

Typical classroom 
practices (n = 448 
students) 

GMRT Reading 
Comprehension 
(nationally normed) 

.07 

3 

L., Anderson, L., et al. 
(2014). Examining 

the effectiveness of 

a multicomponent 
reading comprehension 






Reading 
comprehension 
of expository 
text (researcher 
created) 

-.15 

-6 

intervention in middle 

schools: A focus on 

treatment fidelity. 
Educational Psychology 
Review, 26(3), 425-449. 






Reading 

comprehension 

of narrative text 

(researcher 

created) 

.04 

1 

Little, C. A., McCoach, 

D. B., & Reis, S. (2014). 
Effects of differentiated 

reading instruction on 

student achievement in 

middle school. Journal 

of Advanced Academics, 
25(4), 384-402. 

Schoolwide 

Enrichment 

Model reading 

framework 

Cluster randomized 

controlled trial 

2,150 students 

with 47 teachers 

in four middle 

schools (grades 
6-8) 

Schoolwide 

Enrichment Model 
reading framework 
(n = 27 teachers) 

Typical classroom 
practices (n = 20 
teachers) 

GMRT Reading 
Comprehension 
(nationally normed) 

.07 

3 

Meyer, B. J. F., Wijekumar, 
K., Middlemiss, W., Higley, 
K„ Lei, P.-W., Meier, C., 

Components 
of Intelligent 
Tutoring of 

Randomized 

controlled trial 

111 grade 5 

and 7 students 
participating in a 

Feedback (n = 59 
students) 

No feedback 

(n = 52 students) 

Reading 

Comprehension 

Domain e 

-.02 

-1 

et ai. (2010). Web-based 
tutoring of the structure 
strategy with or without 

the Structure 

Strategy 


computer software 
program titled 
Intelligent Tutoring 

Choice of reading 
passage (n = 56 
students) 

No choice of 
reading passage 
(n = 55 students) 

Reading 

Comprehension 

Domain e 

-.09 

-4 

elaborated feedback 

of choice for fifth- and 
seventh-grade readers. 
Reading Research 

Quarterly, 45(1), 62-92. 



of the Structure 

Strategy 

Simple feedback 
with choice of 

reading passage 
(n = 25 students) 

Simple feedback 

with no choice of 

reading passage 
(n = 27 students) 

Reading 

Comprehension 

Domain e 

-.05 

-2 




Elaborate 

feedback with 

choice of reading 
passage (n = 31 
students) 

Elaborate 

feedback with no 

choice of reading 
passage 

(n = 28 students) 

Reading 

Comprehension 

Domain" 5 

-.09 

-4 


(continued) 
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Table C3. Adolescent literacy programs and practices identified by the review team as having indeterminate effects (continued) 

Full citation 

Program or 
practice 

Research 

design 

Analysis sample 

Intervention 

implementation 

Nature of the 
comparison group 

Student outcome 
assessment 

Effect size a 
(Hedges’ g) 

Improvement 

index b 

Neddenriep, C. E., 

Skinner, C. H., Wallace, 

M. A., & McCallum, E. 
(2009). ClassWide Peer 

ClassWide Peer 

Tutoring 

Single-case 
design: alternating 

treatment 

Four grade 6 

students with 
poor reading skills 

as measured 

Peer tutoring 
(n = 2 cases) 

Reading aloud 
(same students) 

Rate of reading 
comprehension 
(researcher 
developed) 

Strong 

evidence* 1 


Tutoring: Two experiments 
investigating the 
generalized relationship 

between increased oral 
reading fluency and 
reading comprehension. 



by the state 
accountability 

assessment 



Number of reading 

comprehension 

questions 

answered correctly 

(researcher 

developed) 

Moderate 

evidence 11 


Journal of Applied School 
Psychology, 25(3), 

244-269. 




Peer tutoring 
(n = 2 cases) 

Reading aloud 
(same students) 

Number of reading 
comprehension 
questions 
answered correctly 

and rate of 

comprehension 

(researcher 

developed) 

No 

evidence 11 


Ridge, A. D., & Skinner, 

C. H. (2011). Using 
the TELLS prereading 
procedure to enhance 
comprehension levels 
and rates in secondary 
students. Psychology in 
the Schools, 48(1), 46-58 

Title, Examine, 
Look, Look, and 
Setting program 

Single-case 
design: multiple 

baseline 

Three grade 

9 students 

nominated by 
teachers as having 

difficulties in 

reading 

Title, Examine, 

Look, Look, and 
Setting pre- 
reading strategy 
(n = 3 cases) 

Reading aloud 
(same students) 

Number of reading 
comprehension 
questions 
answered correctly 

and rate of 

comprehension 

(researcher 

developed) 

No 

evidence 11 


Vaughn, S., Cirino, P. T., 
Wanzek, J . , Wexler, J., 

Tier II instruction 

emphasizing 

Randomized 

controlled trial 

301 grade 6 
students, more 

Tier II intervention 

(n = 212 students) 

Typical 

instructional 

TAKS (state 
accountability) 

.14 

+6 

Fletcher, J. M., Denton, 

C. D„ etal. (2010). 
Response to intervention 

word recognition, 
vocabulary, 
fluency, and 


than half of the 

sample was 
designated 


practices (n = 114 
students) 

GRADE Reading 
Comprehension 
(nationally normed) 

-.06 

-2 

for middle school 

students with reading 

difficulties: Effects of a 

primary and secondary 
intervention. School 

Psychology Review, 39(1), 

3-21. 

comprehension 


as struggling 

based on state 

accountability test 

scores 



WJ Passage 
Comprehension 
(nationally normed) 

.11 

+4 


(continued) 
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Table C3. Adolescent literacy programs and practices identified by the review team as having indeterminate effects (continued) 

Full citation 

Program or 
practice 

Research 

design 

Analysis sample 

Intervention 

implementation 

Nature of the 
comparison group 

Student outcome 
assessment 

Effect size 3 
(Hedges’ g) 

Improvement 

index b 

Vaughn, S., Wanzek, 

J., Wexler, J., Barth, A., 

Tier II instruction 

emphasizing 

Randomized 

controlled trial with 

421 grade 7 and 8 
struggling readers 

Large-group Tier 

II instruction 

Typical 

instructional 

TAKS (state 
accountability) 

-.18 

-7 

Cirino, P. T., Fletcher, J., 
et al. (2010). The relative 
effects of group size on 

word recognition, 
vocabulary, 
fluency, and 

students assigned 

to one of three 

conditions 


(n = 203 students) 

practices (n = 218 
students) 

GRADE Reading 
Comprehension 
(nationally normed) 

.05 

+2 

reading progress of older 
students with reading 
difficulties. Reading & 
Writing, 23(8), 931-956. 

comprehension 





WJ Passage 
Comprehension 
(nationally normed) 

.03 

+1 



273 grade 7 and 8 
struggling readers 

Small-group Tier II 
instruction (n = 55 

Typical 

instructional 

TAKS (state 
accountability) 

-.22 

-9 





students) 

practices (n = 218 
students) 

GRADE Reading 
Comprehension 
(nationally normed) 

.13 

+5 







WJ Passage 
Comprehension 
(nationally normed) 

.13 

+5 

Vaughn, S., Klingner, 

J. K., Swanson, E. A., 
Boardman, A. G., Roberts, 

Collaborative 

Strategic Reading 

Cluster randomized 

controlled trial 

866 grade 7 and 

8 students in 61 

classes across 

Collaborative 

Strategic Reading 
(n = 34 classes) 

Typical 

instructional 

practices (n = 27 

GMRT 

Comprehension 
(nationally normed) 

.09 

+3 

G., Mohammed, S. S., 
et al. (2011). Efficacy of 
Collaborative Strategic 
Reading with middle 

school students. American 

Educational Research 

Journal, 48(4), 938-964. 



three school 

districts 


classes) 

Test of Silent 

Reading and 
Comprehension 
(nationally normed) 

.07 

+3 

Vaughn, S., Roberts, G., 
Klingner, J. K., Swanson, 

E. A., Boardman, 

Collaborative 

Strategic Reading 

Cluster randomized 

controlled trial 

528 grade 7 and 

8 students in 48 

classes across 

Collaborative 

Strategic Reading 
( n = 26 classes) 

Typical 

instructional 

practices (n = 22 

GMRT - 

Comprehension 
(nationally normed) 

.10 

+4 

A., Stillman-Spisak, 

S. J., et al. (2013). 
Collaborative strategic 
reading: Findings 
from experienced 
implementers. Journal of 

Research on Educational 
Effectiveness, 6(2), 

137-163. 



three school 

districts 


classes) 

Test of Silent 

Reading and 
Comprehension 
(nationally normed) 

0 

0 


(continued) 
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Table C3. Adolescent literacy programs and practices identified by the review team as having indeterminate effects (continued) 


Full citation 

Program or 
practice 

Research 

design 

Analysis sample 

Intervention 

implementation 

Nature of the 
comparison group 

Student outcome 
assessment 

Effect size 3 
(Hedges’ g) 

Improvement 

index b 

Zinn, A., & Courtney, M.E. 

2014. Context matters: 

Experimental evaluation 
of home-based tutoring 
for youth in foster care. 

Children and Youth 

Services Review, 47(3), 

198-204. 

Early State to 
Emancipation 
Preparation one- 
on-one tutoring 

Randomized 

controlled trial 

402 children 

ages 14-18 in 

foster care in Los 

Angeles 

Early Start to 
Emancipation 
Preparation 
tutoring program 
for children in 

foster care, which 

included SRA 

Reading (n = 212 
students) 

Counselors 

conducted typical 
practices (n = 190 
students) 

WJ Passage 
Comprehension 
(nationally normed) 

-.01 

0 


CTBS/5 is Comprehensive Test of Basic Skills (version 5); ITBS is Iowa Test of Basic Skills; TAKS is Texas Assessment of Knowledge and Skills; WJ is Woodcock Johnson Tests of 
Achievement, GMRT is Gates McGinitie Reading Test; GRADE is Group Reading Assessment and Diagnostic Evaluation. 

a. The effect size is a standardized measure of the effect of an intervention on student outcomes representing the change (measured in standard deviations) in an average student's 
outcome that can be expected if that student is given the intervention. 

b. The improvement index is an alternate presentation of the effect size reflecting the change in an average student’s percentile rank that can be expected if the student is given the 
intervention 

c. One of the 12 cases analyzed in this study met standards. 

d. The What Works Clearinghouse (WWC) rates the effects of single-case design studies as demonstrating strong evidence of an effect, moderate evidence of one, or no evidence of 
one. However, rating the effects of an intervention from a single-case design requires the following: a minimum of five single-case design studies must meet WWC standards, the single- 
case design studies must be conducted by at least three different research teams with no overlapping authorship at three different institutions, and the combined number of cases has 
to be at least 20 (U.S. Department of Education, 2014). 

e. The effect size for reading comprehension domain reported in this table reflects the average effects size for the reading comprehension domain. This domain consists of a nationally 
normed assessment of reading comprehension (the Gray Silent Reading Test) and a researcher-developed outcome assessment of reading comprehension. 

Source: Authors' summary of studies. 




Appendix D. Description of the interventions used 
in the 33 studies that the review team found had 
met What Works Clearinghouse evidence standards 


This appendix provides a detailed description of the interventions as described in the pub- 
lished manuscript for the 33 studies that meet What Works Clearinghouse (WWC) evi- 
dence standards with or without reservations, according to the review team. 

Programs and practices demonstrating positive and potentially positive effects 

This section describes interventions that were shown to have positive or potentially posi- 
tive effects on reading comprehension, vocabulary, or general literacy. 

Read Now (Algozzine, 2004). Read Now is an intensive, structured reading intervention 
that combines software; a teacher’s guide with lessons, tips, and activities; and ongoing, 
one-on-one consultant support. It was created to address the five essential components of 
reading identified by the National Reading Panel and includes four key activities: Read To, 
Fluency Practice, Guided Independent Reading, and Small Group or Individual Lessons. 
In the study, Read Now was implemented daily in a 90-minute block for 10 weeks. (Profes- 
sional development associated with the program was not described in the study.) 

Reading Edge (Chamberlain, Daniels, Madden, & Slavin, 2007). Reading Edge, a 
component of the Success for All whole-school reform model, combines four key instruc- 
tional elements: explicit instruction in metacognitive strategy use, cooperative learning, 
goal setting with feedback, and classroom management techniques. All lessons follow a 
set structure: Setting the Stage, Active Instruction, Teamwork, and Time for Reflection. 
To implement Reading Edge, emphasis is placed on grouping students and specific class- 
room management techniques. Students are grouped by their reading level, with each 
group utilizing appropriate text. Students are assessed and regrouped quarterly. Classroom 
management for effective implementation of cooperative learning includes brisk pacing, 
direct instruction of conflict resolution, and academic problem solving. It also includes 
setting cooperation goals; providing extra credit for active listening, encouraging peers, or 
100 percent team participation; randomly selecting team members to provide answers; and 
using discussion routines like Think-Pair- Share and Numbered Heads. 

Teachers using Reading Edge in this study received extensive training and ongoing coaching, 
although the study did not specifically describe the amount and type of training. The 
authors noted that participating teachers did not receive the amount of implementation 
support typical in most schools using Reading Edge: a full-time school-based facilitator and 
the support of the other teachers in the school who are all implementing Reading Edge at 
the same time. 

Story Impressions (Denner, Rickards, & A Ibanese, 2003). Here, two different types of 
reading preview methods, Story Impressions and Content Preview, were implemented in a 
grade 8 general education history course. Story Impressions consisted of a preview sheet for 
the reading comprehension passage (a historical biography) in which the teacher chose 18 
pausal units from the passage that represented the highest level of structural importance. 
The pausal units were reduced to a phrase by retaining only the essential cue words. A 
copy of the pausal unit list is available in the published article. 
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Students read the phrases and wrote a history connecting all the clues in the same order 
to formulate a written guess about what might happen in the passage. Students took some 
20 minutes to complete their written responses. The Content Preview method used the 
same 18 preview units, adding vocabulary definitions and questions to promote interest 
and activate prior knowledge. Professional development consisted of scripted instructions 
and a two-hour discussion of procedures with the researchers. 

Embedded Story Structure Routine (Faggella-Luby, Schumaker, & Deschler, 2007). 
The Embedded Story Structure Routine consists of three strategies: selfiquestioning at 
pre-reading, story structure analysis during reading, and summary writing after reading. 
The three strategies are implemented using a graphic organizer to facilitate collaboration 
between the teacher and students and between students and peers. The self-questioning 
strategy involves seven questions (who, what, when, where, which, how, and why) related 
to the critical components of narrative story structure. (This study was conducted in a 
summer school program for rising grade 9 students who were found, based on their grade 8 
test scores, to be at risk.) After answering the questions, students complete a story structure 
diagram and draw picture cues. Finally, they write a four-sentence summary that includes 
the critical elements of the story. Students are given mnemonic devices to remember the 
strategies. An illustration of the Embedded Story Structure Routine is available in the 
publication. 

In this study, instruction was delivered for a total of eight stories over nine days during a 
summer program, with daily instruction ranging from 90 minutes to 120 minutes for a total 
of 17 hours of instruction. Students were in groups of 12-14. The teacher was a doctoral 
student in special education and one of the authors of the study, so no additional training 
was provided. 

QeoLiteracy (Hinde, Popp, Dorn, E kiss, Mater, Smith, et al., 2007). GeoLiteracy is a 
package of 85 lessons (drawn from Kindergarten through grade 8) that teaches geography 
in the context of practicing reading and writing skills. GeoLiteracy lessons emphasize the 
following reading skills: cause and effect, sequencing, main idea, summarizing, drawing 
conclusions or inferences, following directions, and reading/interpreting graphic displays. 
In this study three to five GeoLiteracy lessons were selected from a predetermined set 
of lessons and implemented during language arts or social studies sessions. Each lesson 
spanned from one to three class periods. Many of the intervention teachers participat- 
ing in this study were teacher consultants for GeoLiteracy, and many other intervention 
teachers received consultation from the teacher consultants at their schools. 

Read 180 Enterprise (Kim, Capotosto, Hartry, & Fitzgerald, 2011). Read 180 con- 
sists of three rotations in a 90-minute session: whole-group teacher-directed instruction to 
build background and model fluency and comprehension strategies; individualized comput- 
er-assisted instruction on decoding, fluency, vocabulary, and comprehension; independent 
reading practice with high-interest text; and teacher-directed instruction in small homo- 
geneous groups. In this study, the typical Read 180 program was adapted to a 60-minute 
session and implemented four days per week for 23 weeks in an after-school program for 
students who performed poorly on the state outcome assessment. The adaptation of Read 
180 in this study included only two of the three rotations at each session. The teach- 
ers implementing Read 180 received preservice and in-service training and were certified 
Read 180 instructors. 
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Tutoring (Rothman & Henderson, 2011). This tutoring program was created by staff at a 
New Jersey school in which two test-taking programs were combined and provided to small 
groups of students (student-teacher ratio of 4:1). They met twice a week after school in 
90-minute sessions for some 16 weeks, resulting in a total of 48 hours of tutoring. Students 
completed one chapter from the Preparing for the New Jersey GEPA book per week. In 
addition, tutors were instructed to reinforce the strategies found in the Standard Solu- 
tions program. The Standard Solutions test-taking strategies program includes instruction 
on identifying and eliminating options on multiple-choice tests, on properly completing 
open-ended questions, and on time management. During the program, tutors attended 
four meetings with the researchers to discuss tutoring content and any potential problems. 
In addition, the tutors provided weekly reports to the students’ classroom teacher. 

Student Team Reading and Writing (Stevens, 2003). The components of the Student 
Team Reading and Writing program included cooperative-learning classroom processes; 
a literature anthology for high-interest reading material; explicit instruction in reading 
comprehension; integrated reading, writing, and language arts instruction; and a writing 
process approach to language arts. The reading part of the program included three core 
elements: literature-related activities, direct instruction in reading comprehension strate- 
gies, and selection-related writing. All activities were conducted in heterogeneous student 
teams and followed the same cycle: teacher presentation, team practice, independent prac- 
tice, peer pre-assessment, and individual accountability. This program was implemented 
for nine months to students in grades 6-8 attending two middle schools in a large urban 
school district. 

Multicomponent social studies instruction (Vaughn, Martinez, Linan-Thompson, Reu- 
tehuch, Carlson, & Francis, 2009). Multicomponent social studies instruction is com- 
posed of explicit vocabulary and concept instruction, the use of brief videos and purposeful 
discussion to build concepts, the use of graphic organizers and other writing activities to 
build comprehension and vocabulary through writing, and structured paired grouping. 

During group activities, students with limited English proficiency were paired with fluent 
English speakers. Structured pairing was incorporated to improve instruction: It increased 
students’ access to and practice with the language associated with content-area instruc- 
tion, providing an interactive and motivating structure for reading and discussing ideas 
and concepts; it also engaged English learners in discussions by providing a scaffold and 
practice. Vocabulary instruction included selecting words to improve students’ academic 
language, providing students with opportunities to encounter new words in texts or video 
clips, and using graphic organizers to reinforce word meanings and show associations 
between Spanish and English words. In this study grade 7 students received instruction 
during their regular social studies class. It was implemented for 50 minutes per day, five 
days per week, for some 9-12 weeks. 

Teachers received a one-day professional development workshop on implementing the 
treatment practices. Each teacher also got all the necessary materials to implement the 
instruction, in-class support from the researchers, and one-on-one coaching. 

Promoting Acceleration of Comprehension and Content Through Text (PACT; Vaughn, 
Swanson, Roberts, Wanzek, S tillrnan-Spisak, Solis, et al., 2013). PACT is an instruc- 
tional practice delivered by teachers that includes five components focusing on improving 
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understanding during text reading and providing opportunities for students to integrate 
newly learned information with previously learned information. The five components 
include a comprehension canopy that contains motivational and background-building 
components. For background building, new learning is consistently tied back to previous 
learning; introduction and review of key terms; teacherded discussions to help students’ 
focus on key information in the text; tearmbased learning that includes completion of 
comprehension checks and discussions within heterogeneous groups; and tearmbased 
learning debates requiring students to use textual evidence, critical thinking, synthesis 
of group perspectives, and presentation of final written product to the class. In this study, 
teachers implemented PACT in 50-54 minute sessions in three 10-day session blocks over 
a six- to eight-week period in their grade 8 social studies classes. Beforehand, to facilitate 
implementation, teachers received some 10 hours of training, subsequent in-class visits, 
and teacher planning sessions. 

Dramatic Impact (Walker, Tabone, & Weltsek, 2011). Teachers worked with teach- 
ing artists to infuse aspects of theater arts, such as theater games, scenery design, process 
drama, improvisation, script writing, and enactment, into their language arts instruction 
around the core novels used in the district curricula. In this approach, 40 drama-based 
lesson plans linked to the mandated literary texts for students in grades 6 and 7 were 
implemented in language arts classrooms over the academic year. Professional develop- 
ment was not needed because the teaching artist worked directly in the classroom along- 
side the teacher. 

CareerStart (Woolley, Rose, Orthner, Akos, & Jones-Sanpei, 2013). CareerStart is a 
program that advances the relevance of instructional content through the use of occupa- 
tional examples infused into the standard curriculum across math, science, social studies, 
and language arts. The program includes 10 lessons for each core subject that content-area 
teachers can use to illustrate important concepts and lessons in their courses. In addition, 
teachers are encouraged to infuse career examples and problems across their teaching. 
Students received this intervention during all three years of middle school (grades 6-8). 
Professional development was available for participating teachers but was not explicitly 
described in the study. 

Programs and practices demonstrating mixed effects 

This section describes interventions with studies that found mixed effects on reading com- 
prehension, vocabulary, or general literacy outcomes. 

Reading Apprenticeship (Corrin, Somers, Kemple, Nelson, & Sepanik, 2008; Qreen- 
leaf, Hanson, et al., 2011; Qreenleaf, Litman, et al., 2011; Kemple, Corrin, Nelson, 
Salinger, Herrmann, Drummond, et al., 2008). Four studies investigated the impact of 
Reading Apprenticeship on reading comprehension and vocabulary. This section describes 
the four and any differences in implementation that occurred between them. 

Corrin et al. (2008) and Kemple et al. (2008) implemented Reading Apprenticeship, an 
established supplemental literacy intervention that was developed for grade 9 students 
whose reading skills were below or far below grade level. Reading Apprenticeship, distrib- 
uted by WestEd, focuses on student motivation and engagement, reading fluency, vocabu- 
lary, comprehension, phonics and phonemic awareness (for students who could still benefit 
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from instruction in these areas), and writing. Teachers demonstrate the skills of master 
readers and incorporate five aspects of comprehension instruction: metacognitive con- 
versation, silent sustained reading, language study, content/themes, and writing. Instruc- 
tional routines include thinking aloud, talking to text, metacognitive journals, and daily 
warmups. In these studies the programs were slightly modified and implemented as a sup- 
plemental class that replaced another elective class. This elective was taken in addition to 
students’ regular English language arts classes. The course was scheduled for a minimum of 
225 minutes per week for a full school year in classes of 12 to 15 students. 

The programs’ developers tailored professional development and coaching strategies to 
high school teachers who lacked reading instruction credentials. Professional development 
workshops were conducted for the teachers before the school year began and in booster 
sessions throughout the year. Coaches made three two-day visits to each participating 
teacher during the year. 

In one study (Greenleaf, Hanson, et al, 2011) teachers received a total of 10 days of pro- 
fessional development before the start of the school year, mid-year, and at year-end. Expe- 
riential training methods were used during professional development so teachers could 
experience Reading Apprenticeship’s instructional methods in the same way students do. 
The instructors collected information through interviews and emails to plan the mid-year 
and year-end professional development sessions. The study paid for teacher travel to the 
sessions, along with an honorarium, provision of substitute teachers, and up to $500 for 
purchasing additional instructional reading materials. In addition, a listserv was available 
to facilitate interactions among implementing teachers. 

The report also included several outcomes that did not meet WWC group design stan- 
dards: those for the U.S. history classes, those on the California Standards Test for biology, 
and all subgroup analyses. 

Content Literacy Continuum ( Corrin , Lindsay, Somers, Myers, Meyers, Condon, 
et al., 2012). This intervention combines instructional routines with learning strategies 
developed and tested by the University of Kansas Center for Research on Learning. The 
Content Literacy Continuum’s (CLC) levels of support range from schoolwide backing to 
promoting student literacy and content learning (analogous to the tiered response to inter- 
vention practices). First, all core content teachers (English language arts, science, social 
studies, math) use instructional routines and model learning strategies with content-area 
text. The instruction is intended to be sequenced, scaffolded, and include multiple modal- 
ities. Second, trained reading teachers provide more intensive instruction targeted to each 
struggling adolescent reader using a curriculum called Fusion Reading, which incorporates 
explicit instruction in foundational decoding, fluency, and comprehension skills (Hock, 
Brasseur, & Deshler, 2008). In this study, CLC was given to the same students in grades 9 
and 10. 

Each school had an external CLC site coordinator who was trained at the University of 
Kansas Center for Research on Learning and was responsible for conducting professional 
development, setting up Reading Fusion classes, and consulting with the school’s Literacy 
Leadership Team. There were 1. 6-1.9 days of professional development per month for six 
to seven months. 
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Programs and practices demonstrating an indeterminate effect 


This section describes interventions with studies that found indeterminate effects on 
reading comprehension, vocabulary, or general literacy. 

Homework methods (Alber, Nelson, & Brennan, 2002). Instead of a typical homework 
assignment, students in a grade 9 general education social studies class got a standard 
review questions study method and a structured worksheet to accompany their homework 
reading text. The standard review questions study method consisted of four to six summary 
questions. The structured reading worksheet was made up of 12 to 24 fill-in-the-blank 
questions paraphrased from the text. The social studies teacher assigned the homework 
four days per week for three weeks. Because of the nature of the instruction, professional 
development for the teachers was not needed. 

Fluency building (Al Under, Dunse, Brunken, & Obermiller-Krolikowski, 2001). Stu- 
dents met with a teacher to select a reading strategy for them to focus on for a 10 -week 
period. Strategies included using morphological strategies to understand word roots and 
affixes, pausing at punctuation, reading with expression, correcting errors by self-monitor- 
ing, and tracing print with a forefinger. Teachers selected each student’s intervention based 
on observation and input from the students themselves. Students received a bookmark 
with their selected strategy written on it for use while reading text. Professional develop- 
ment was not described in detail in this study. 

Singlesex classrooms (Belcher, Frey, & Yankeelov, 2006). Students participating in sin- 
gle-sex classrooms for one school year were compared with those participating in co-edu- 
cational classes in the same school. No professional development was needed for this study. 

Fast ForWord Language (Borman, Benson, & Overman, 2009). The Fast ForWord Lan- 
guage computer software program provides exercises on phonological awareness, language 
processing speed, sequencing, vocabulary, and language comprehension. The program 
requires 100 minutes of training per day, five days a week, for four to eight weeks under the 
supervision of Fast ForWord-trained clinicians or educators. Students participated in the 
study as a pullout/supplemental program. More than half of the students attended for the 
recommended minimum standard of 20 days, and 43 percent met all three of the develop- 
ers’ criteria for successful completion. The program was implemented by trained educators, 
and no further professional development was needed. 

Xtreme Reading (Corrin, Somers, Kemple, Nelson, & Sepanik, 2008; Kemple, Corrin, 
Nelson, Salinger, Herrmann, Drummond, et al., 2008). Both studies investigated 
Xtreme Reading, an established supplemental literacy intervention developed for grade 9 
students whose reading skills were either below or far below grade level. Xtreme Reading, 
which was developed by the University of Kansas Center for Research on Learning, 
focuses instruction on student motivation and engagement, reading fluency, vocabulary, 
comprehension, phonics and phonemic awareness (for students who could still benefit 
from instruction in these areas), and writing. This program also helps students adopt the 
strategies and routines used by proficient readers. Xtreme Reading utilizes detailed daily 
lesson plans with explicit instruction in seven reading strategies using a prescribed eight- 
stage instructional routine. The strategies are LINCS Vocabulary Routine, Word Mapping, 
Word Identification, Self-Questioning, Visual Imagery, Paraphrasing, and Inferencing. The 
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teachers use such methods as describe, model, verbal practice, guided practice, paired prac- 
tice, independent practice, differentiated instruction, and integration and generalization. 

The programs’ developers tailored professional development and coaching strategies to 
high school teachers who lacked reading instruction credentials. Professional development 
workshops were conducted for the teachers before the school year and in booster sessions 
throughout the year. Coaches made three two-day visits to each participating teacher 
throughout the year. 

Thinking Reader (Drummond, Chinen, Duncan, Miller, Fryer, Zmach, et al., 2011). 
Thinking Reader employs reciprocal teaching in three phases: teacher-directed pre-read- 
ing discussion, use of the software, and teacher-directed post-reading activities. Pre-reading 
activities include modeling strategy use and summarizing previous sessions. The software 
provides the text, a dictionary, and opportunities for the student to use one of seven strat- 
egies to think about the text: summarize, predict, question, clarify, visualize, feel, or reflect. 
The teacher monitors students’ performance and can customize the level of support pro- 
vided to each student. Post-reading activities consist of student-teacher discussion or an 
activity to demonstrate comprehension. Program publishers recommended that teachers 
use the Thinking Reader program for 110-165 minutes per week for three different novels, 
with each novel spanning four to six weeks and breaks included in between. Student par- 
ticipation in each novel was less than recommended ranges. Teachers participated in two 
training sessions lasting six hours each. They also received individual coaching sessions 
totaling 7.5— 8.5 hours apiece. 

Renzulli Learning (Field, 2010). Renzulli Learning is an online educational learning 
system and profiler designed to match student interests, learning styles, and expression 
styles with challenging enrichment resources. In this study classes of students in an ele- 
mentary and a middle school used Renzulli Learning for two to three hours per week for 
16 weeks. In the middle school this took place in the Technology Connections class. Each 
student completed the Renzulli Learning Profiler, an online questionnaire about students’ 
interests, abilities, learning styles, and modes of expression. Upon completing the Profiler, 
students had access to their own Enrichment Database on Renzulli Learning, as well as a 
database of more than 30,000 education resources and activities such as virtual field trips, 
real field trips, creativity training, independent projects and studies, contests and compe- 
titions, websites, fiction books and e-books, nonfiction books and e-books, how-to books 
and e-books, summer programs, online activities and classes, research sites, and videos and 
DVDs. 

The study also examined the effects for a subgroup — gifted students. Substantively import- 
ant but not statistically significant effects were found for gifted students on the reading 
comprehension and reading fluency outcomes. 

Comprehension Circuit Training Intervention (Fogarty, Oslund, Simmons, Davis, 
Simmons, Anderson, et al., 2014). Comprehension Circuit Training (CCT) is a multi- 
component framework of instructional practices to facilitate adolescents’ comprehension 
of complex text. It was developed around an exercise theme, with students progressing 
through stations — the way people move through circuit training when they exercise. CCT 
was organized in two phases: short fiction and expository. In phase one, practices and strat- 
egies were introduced using two short fiction texts. The routine and instructional practices 
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were then transferred to five teacher-selected short fiction texts. In phase two, the routine 
and practices were transferred to three teacher-selected expository texts. Recommended 
language was provided for the first two texts in both phases. After two full days of pro- 
fessional development, teachers were asked to implement, over the course of 12 weeks, 36 
lessons lasting 50 minutes each. The researchers also provided additional individual and 
group professional development. 

School-wide Enrichment Model reading framework (Little, M cCoach, & Reis, 2014). 
The Schoohwide Enrichment Model reading framework study was completed within a 
reading class: students read independently while teachers conducted individual confer- 
ences. During the conference, the teacher determined if the student’s self-selected book 
was appropriately challenging, provided differentiated instruction in reading skills and 
strategies, and discussed higher level questions about the text. Implementation consisted 
of three phases. Phase 1 included read-alouds and a brief discussion on a large variety of 
text. Phase 2 included independent reading and five- to seven-minute individual confer- 
ences: each student would meet with the teacher at least once every two weeks. Phase 3, 
implemented during the second half of the school year, included project-based activities 
chosen based on students’ interest. Reading and language arts teachers implementing the 
School-wide Enrichment Model reading framework were expected to conduct the program 
for 40 to 45 minutes per day, every day. Teachers participated in a one-day professional 
development session, one follow-up session mid-year, and support from project staff who 
visited classrooms once every two to three weeks. 

Components of Intelligent Tutoring of the Structure Strategy (Meyer, Wijekumar, M id- 
dlemiss, Higley, Lei, Meier, et al., 2010). All students in this study were participating 
in the Intelligent Tutoring of the Structure Strategy online program. Its system allows 
students to interact with an animated agent/tutor to learn and practice the strategy and 
receive immediate feedback. Each tutor-tutee interaction is unique and adapts to the stu- 
dent’s performance. There are 95 lessons (65 regular plus 30 alternative-choice lessons) 
that were designed through consultation with teachers, motivation experts, and students. 

The study evaluated two features of the Intelligent Tutoring of the Structure Strategy 
system: feedback and choice of passage. Students were provided with simple or elaborated 
feedback and were given either a choice or no choice in reading passage. Intelligent Tutor 
is administered via the computer, and professional development was not needed to imple- 
ment the study. 

ClassWide Peer Tutoring (N eddenriep, Skinner, Wallace, & McCallum, 2009). In 
this study, ClassWide Peer Tutoring was implemented with pairs of students reading to 
each other using passages from the Timed Readings series. Tutoring sessions occurred in a 
pullout setting in the middle school and were implemented by the researchers. The tutee 
would begin by reading the first half of a passage aloud while the tutor provided feedback, 
corrected errors, and awarded points to the tutee. The tutor and tutee then switched roles 
and the new tutee read the same first half of a passage. Each reading lasted five minutes 
and students were assessed on a researcher-developed test of reading comprehension on a 
different passage. 

Title, Examine, Look, Look, and Setting (TELLS; Ridge & Skinner, 2011). The Title, 
Examine, Look, Look, and Setting (TELLS) procedure is a comprehension strategy taught 
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to students for use before reading a text. The student first looks at the title to form clues 
about what will be covered, then examines the passage for clues about the content, looks 
for important, often-used words, looks for unknown words and finds their meaning, and 
skims the passage for clues about the setting. In the study the TELLS strategy was taught 
to students in a one-on-one setting by the experimenter. Sessions, lasting 10 to 30 minutes, 
were held three days per week for some two and a half weeks. The worksheet used to guide 
the TELLS sessions is included in the publication. 

Tier II intervention ( Vaughn , Cirino, Wunzek, Wexler, Fletcher, Denton, et al., 2010). 
The teachers of all students in both the treatment and comparison groups of this study 
participated in professional development designed to enhance the quality of the core 
reading instruction (also called Tier I instruction). The research team provided profession- 
al development on evidence-based practices for teaching vocabulary and comprehension 
to the content-area teachers of all grade 6 students. Teachers attended a six-hour profes- 
sional development session at the beginning of the school year and then met in study 
groups at their respective schools roughly once a month throughout the year. Study groups 
consisted of interdisciplinary teams in six of the schools, though one school framed study 
groups by department area. In-classroom coaching was provided on request. During the 
monthly study group sessions, teachers worked with a facilitator to apply these strategies 
while planning lessons in their own content areas. 

The Tier II intervention was an extensive yearlong intervention, with students placed in 
homogeneous groups to the extent class schedules allowed. The researcher-provided inter- 
vention included three phases: partner reading to build fluency, word study using lessons 
from REWARDS Intermediate, explicit vocabulary and reading comprehension instruc- 
tion, and comprehension lessons from REWARDS Plus. Phase 1 consisted of some 25 
lessons taught over seven to eight weeks and emphasized word study and fluency skills. In 
Phase II, instruction stressed vocabulary and comprehension, with additional instruction 
and practice provided for applying the word study and fluency skills and strategies learned 
in Phase I. Phase II lessons occurred over 17 to 18 weeks, depending on students’ progress. 
Phase III, which continued over some 8 to 10 weeks, kept the emphasis on vocabulary and 
comprehension. 

Collaborative Strategic Reading (Vaughn, Klingner, Swanson, Boardman, Roberts, 
Mohammed, et al., 2011; Vaughn, Roberts, Klingner, Swanson, Boardman, Still- 
man-Spisak, et al., 2013). Collaborative strategic reading is an instructional method for 
reading comprehension structured around the narrative and expository text already used 
by a school. Students learn a variety of comprehension strategies, including main idea, 
questioning, previewing, “click and clunk,” “get the gist,” and wrap-up. Students practice 
strategies independently and as a leader in a collaborative peer group. Among the ele- 
ments are explicit instruction by the teacher, implementing procedural strategies, using 
interactive groups or partners, and providing opportunities for interactive dialogue among 
students and between teachers and students. Additional details for the intervention are 
available in the publication and at www.meadowscenter.org. 

Researchers in Vaughn et al. (2011) provided teachers with initial training in collabo- 
rative strategic reading over three days. Each teacher received all necessary materials to 
implement the treatment, including sample lessons, examples of reading materials, and 
overheads. The researchers also provided subsequent professional development on three 
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occasions over the implementation year (about 90 minutes per meeting) to enhance and 
refine implementation and use of collaborative strategic reading in their classrooms. Each 
participating teacher was assigned a coach who was initially present one to two times per 
month during a treatment class and then less so over the course of the intervention. Stu- 
dents in grades 7 and 9 in the treatment classes received the intervention during their 
regularly scheduled English language arts class. Teachers were asked to implement the 
intervention for 50 minutes a day, two days per week, for approximately 18 weeks. Teachers 
said the number of sessions they implemented ranged from 24 to 48. 

The next study conducted by Vaughn et al. (2013) reports the findings of the second year 
of implementation of collaborative strategic reading by the same English language arts or 
reading teachers who participated in the previous study by Vaughn et al. (2011). Teachers 
not only received professional development and implementation support during the first 
year of implementation, but also a full day of professional development as a refresher, dis- 
cussion of the previous year’s results with suggestions for greater effectiveness, and three 
90-minute follow-up professional development sessions throughout the year. Each teacher 
also had a coach available who provided modeling, side-by-side teaching, and observation 
with feedback. Teachers implemented collaborative strategic reading 50 minutes per day, 
two days per week, for 18 weeks. 

Large-group and small-group intervention in middle schools (Vaughn, Wanzek, Wexler, 
Barth, Cirino, Fletcher, et al., 2010). The same Tier I and Tier II instructional practices 
conducted with another sample (see Vaughn, Cirino, Wanzek, Wexler, Fletcher, Denton, 
et al., 2010) were used in this study. Here, some students participated in the Tier II instruc- 
tion as part of the small-group condition consisting of some five students. Students partic- 
ipating in the large-group condition received the same Tier II instructional practices — but 
in a group of 12 to 15 students. Both the small-group and large-group conditions were 
compared with students who participated in the Tier I instruction and any other sup- 
plemental instruction provided by the school. Instruction in both Tier II conditions was 
delivered by certified teachers hired by the researchers, and instruction occurred daily for 
45 to 50 minutes during a regular class period. The Tier II teachers participated in about 
60 hours of professional development before teaching students and received nine hours 
of professional development throughout the year. The researchers also provided ongoing 
feedback through coaching and conducted biweekly meetings. 

Early Start to Emancipation Preparation tutoring (Z inn & Courtney, 2014). Early 
Start to Emancipation Preparation is a one-on-one at-home tutoring program provided to 
foster children ages 14-18. Tutors met with students twice per week for a total of 17 hours 
of reading instruction, which consisted of the SRA Reading 3A curriculum and Hough- 
ton-Mifflin curricula for spelling and vocabulary. Tutors were undergraduate or graduate 
students who received one day of training before tutoring and follow-up trainings twice per 
year. However, researchers noted that 12 percent of control students mistakenly received 
the tutoring, while 38 percent of the students assigned to the treatment group did not 
actually get the tutoring. 
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Notes 


1. Although this review relied heavily on WWC protocols, procedures, and standards and 
the reviews were conducted by WWC-certified reviewers, this is not a WWC product. 
To be a WWC product, the reviews must have been conducted by WWC contractors. 

2. Each of the 12 programs and practices identified as demonstrating positive or poten- 
daily positive effects is represented by one study citation. However, one of the study 
citations (Vaughn et al., 2009) includes findings from two studies that both demon- 
strated statistically significant effects, so the program was assigned a positive effect 
rating. 

3. Reading comprehension tests assess the student’s ability to read a passage and answer 
questions based on the material in the text. Prior knowledge of the material is not 
required to complete the assessment successfully. Content-area knowledge tests that 
assess an individual’s preexisting understanding of the facts, theories, and related mate- 
rials in that area (such as math, science, or social studies) are not eligible outcomes. 

4. Although the reviewers were certified to conduct WWC reviews, this project was not 
sponsored by WWC. 
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